2024-04-24T22:27:19.053 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T22:27:19.054 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135 --verbose --owner scheduled_teuthology@teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135/orig.config.yaml 2024-04-24T22:27:19.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T22:27:19.195 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135 branch: squid description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672135' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: false mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - overall HEALTH_ - \(MON_DOWN\) sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFOUYB/JG2fAyFqtbBUdOcyVMq3GimtEw9l8u24AALYq0jtgM3DKmD8AYRbbZ58s3yUDHqiyZktcO+CGoMuyWNE= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNuj/FFs27X8PNYyAYlZ7ag45pEvHd0Kr6FrNHG/Nr4cnOjY0h9VgFE20HO2RsC2/N7/A9mQ4R81JSR417+VSzY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T22:27:19.195 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T22:27:19.196 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T22:27:19.196 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T22:27:19.198 INFO:teuthology.task.internal:Checking packages... 2024-04-24T22:27:19.221 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T22:27:19.221 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:27:19.221 INFO:teuthology.packaging:ref: None 2024-04-24T22:27:19.221 INFO:teuthology.packaging:tag: None 2024-04-24T22:27:19.221 INFO:teuthology.packaging:branch: squid 2024-04-24T22:27:19.221 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:27:19.221 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T22:27:19.421 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T22:27:19.423 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T22:27:19.434 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T22:27:19.434 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T22:27:19.454 INFO:teuthology.task.internal:Saving configuration 2024-04-24T22:27:19.475 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T22:27:19.486 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T22:27:19.512 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:21:44.510992', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFOUYB/JG2fAyFqtbBUdOcyVMq3GimtEw9l8u24AALYq0jtgM3DKmD8AYRbbZ58s3yUDHqiyZktcO+CGoMuyWNE='} 2024-04-24T22:27:19.537 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:21:44.509216', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNuj/FFs27X8PNYyAYlZ7ag45pEvHd0Kr6FrNHG/Nr4cnOjY0h9VgFE20HO2RsC2/N7/A9mQ4R81JSR417+VSzY='} 2024-04-24T22:27:19.537 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T22:27:19.548 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-24T22:27:19.549 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-24T22:27:19.549 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T22:27:19.621 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f859633f0d0>, signals=[15]) 2024-04-24T22:27:19.621 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T22:27:19.630 INFO:teuthology.task.internal:Opening connections... 2024-04-24T22:27:19.630 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:27:19.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:27:19.712 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:27:19.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:27:19.788 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T22:27:19.799 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-04-24T22:27:19.818 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-04-24T22:27:19.818 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-04-24T22:27:19.877 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:27:19.878 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:27:19.879 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:27:19.879 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-04-24T22:27:19.899 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-04-24T22:27:19.917 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-04-24T22:27:19.917 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-04-24T22:27:19.973 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream" 2024-04-24T22:27:19.973 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:27:19.974 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:27:19.975 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-04-24T22:27:19.999 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T22:27:20.013 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T22:27:20.027 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T22:27:20.028 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:27:20.030 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:27:20.046 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T22:27:20.057 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T22:27:20.057 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:27:20.088 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:27:20.118 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T22:27:20.177 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T22:27:20.200 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T22:27:20.201 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-24T22:27:20.201 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-24T22:27:20.201 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:27:20.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:27:20.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-24T22:27:20.462 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T22:27:20.463 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T22:27:20.463 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-04-24T22:27:20.481 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:27:20.481 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T22:27:20.481 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T22:27:20.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum upgrade -y linux-firmware 2024-04-24T22:27:22.790 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-24T22:27:25.541 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-24T22:27:30.534 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.2 MB 00:01 2024-04-24T22:27:32.034 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-04-24T22:27:33.061 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-24T22:27:38.186 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-24T22:27:40.225 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-24T22:27:40.230 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-24T22:27:40.231 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-24T22:27:40.345 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T22:27:40.346 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T22:27:40.565 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T22:27:40.566 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T22:27:40.569 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi047.front.sepia.ceph.com to complete... 2024-04-24T22:27:52.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24 22:27:52 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T22:27:52.704 INFO:teuthology.orchestra.run.smithi047.stderr:FINISHED --2024-04-24 22:27:52-- 2024-04-24T22:27:52.704 INFO:teuthology.orchestra.run.smithi047.stderr:Total wall clock time: 12s 2024-04-24T22:27:52.704 INFO:teuthology.orchestra.run.smithi047.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-24T22:27:52.707 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T22:27:52.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T22:27:54.899 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying... ######################################## 2024-04-24T22:27:55.021 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing... ######################################## 2024-04-24T22:27:55.118 INFO:teuthology.orchestra.run.smithi047.stdout:Updating / installing... 2024-04-24T22:28:03.462 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T22:28:43.249 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T22:28:43.249 DEBUG:teuthology.orchestra.run.smithi047:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T22:28:43.339 INFO:teuthology.orchestra.run.smithi047.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T22:28:43.339 INFO:teuthology.orchestra.run.smithi047.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T22:28:43.339 INFO:teuthology.orchestra.run.smithi047.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T22:28:43.339 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T22:28:43.339 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T22:28:43.339 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T22:28:43.339 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T22:28:43.340 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T22:28:43.341 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T22:28:43.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T22:28:43.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T22:28:43.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T22:28:43.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T22:28:43.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T22:28:43.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T22:28:43.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T22:28:43.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T22:28:43.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T22:28:43.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T22:28:43.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T22:28:43.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T22:28:43.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T22:28:43.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T22:28:43.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T22:28:43.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T22:28:43.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T22:28:43.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T22:28:43.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T22:28:43.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T22:28:43.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T22:28:43.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T22:28:43.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T22:28:43.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T22:28:43.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T22:28:43.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T22:28:43.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T22:28:43.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T22:28:43.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T22:28:43.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T22:28:43.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T22:28:43.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T22:28:43.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T22:28:43.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T22:28:43.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T22:28:43.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T22:28:43.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T22:28:43.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T22:28:43.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T22:28:43.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T22:28:43.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T22:28:43.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T22:28:43.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T22:28:43.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T22:28:43.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T22:28:43.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T22:28:43.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T22:28:43.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T22:28:43.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T22:28:43.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T22:28:43.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T22:28:43.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T22:28:43.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T22:28:43.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T22:28:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T22:28:43.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T22:28:43.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T22:28:43.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T22:28:43.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T22:28:43.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T22:28:43.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T22:28:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T22:28:43.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T22:28:43.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T22:28:43.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T22:28:43.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T22:28:43.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T22:28:43.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T22:28:43.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T22:28:43.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T22:28:43.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T22:28:43.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T22:28:43.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T22:28:43.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T22:28:43.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T22:28:43.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T22:28:43.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T22:28:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T22:28:43.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T22:28:43.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T22:28:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T22:28:43.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T22:28:43.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T22:28:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T22:28:43.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T22:28:43.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T22:28:43.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T22:28:43.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T22:28:43.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T22:28:43.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T22:28:43.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T22:28:43.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T22:28:43.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T22:28:43.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T22:28:43.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T22:28:43.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T22:28:43.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T22:28:43.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T22:28:43.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T22:28:43.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T22:28:43.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T22:28:43.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T22:28:43.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T22:28:43.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T22:28:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T22:28:43.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T22:28:43.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T22:28:43.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T22:28:43.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T22:28:43.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T22:28:43.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T22:28:43.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T22:28:43.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T22:28:43.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T22:28:43.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T22:28:43.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T22:28:43.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T22:28:43.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T22:28:43.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T22:28:43.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T22:28:43.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T22:28:43.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T22:28:43.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T22:28:43.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T22:28:43.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T22:28:43.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T22:28:43.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T22:28:43.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T22:28:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T22:28:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T22:28:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T22:28:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T22:28:43.475 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T22:28:43.477 DEBUG:teuthology.orchestra.run.smithi047:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T22:28:43.502 INFO:teuthology.orchestra.run.smithi047.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout:for file in vmlinuz System.map config; do 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout: fi 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout:done 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T22:28:43.503 INFO:teuthology.orchestra.run.smithi047.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T22:28:43.504 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-04-24T22:28:43.504 INFO:teuthology.orchestra.run.smithi047.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T22:28:43.504 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T22:28:43.504 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T22:28:43.504 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-04-24T22:28:43.505 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-04-24T22:28:43.586 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.06 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 68.el9 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Group : Unspecified 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 8264254 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : builder@centos.org 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-04-24T22:28:43.587 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T22:28:43.588 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-04-24T22:28:43.588 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-04-24T22:28:43.588 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:28:43.588 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T22:28:43.588 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T22:28:43.588 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T22:28:43.588 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-04-24T22:28:43.588 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:28:43.588 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T22:28:43.589 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T22:28:43.589 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T22:28:43.589 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T22:28:44.168 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-04-24T22:28:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T22:28:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-04-24T22:28:45.626 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-24T22:28:45.655 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.ugSVgR9dCN 2024-04-24T22:28:45.655 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ugSVgR9dCN 2024-04-24T22:28:45.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.ugSVgR9dCN 2024-04-24T22:28:45.860 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.ugSVgR9dCN is 6KB 2024-04-24T22:28:45.908 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.ugSVgR9dCN 2024-04-24T22:28:45.923 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T22:28:45.984 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T22:28:45.985 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T22:28:45.985 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T22:28:45.985 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T22:28:45.986 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T22:28:46.156 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-04-24T22:29:16.159 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-24T22:29:16.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-24T22:29:16.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:29:34.626 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-04-24T22:29:43.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-24T22:29:43.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:30:43.630 DEBUG:teuthology.orchestra.remote:timed out 2024-04-24T22:30:55.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-24T22:30:55.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:30:55.974 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-04-24T22:30:56.674 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-24T22:30:56.675 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T22:30:56.675 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-04-24T22:30:56.731 INFO:teuthology.orchestra.run.smithi047.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T22:30:56.732 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T22:30:56.732 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T22:30:56.732 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T22:30:57.735 DEBUG:teuthology.parallel:result is None 2024-04-24T22:30:57.736 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T22:30:57.744 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T22:30:57.744 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:30:57.752 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:30:57.804 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T22:30:57.812 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T22:30:57.824 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T22:30:57.824 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:30:57.831 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:30:57.899 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T22:30:57.907 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T22:30:57.907 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:30:57.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:30:57.924 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:30:57.932 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:30:58.015 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:30:58.056 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:30:58.057 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:30:58.116 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:30:58.121 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T22:30:58.128 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T22:30:58.128 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:30:58.135 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:30:58.223 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T22:30:58.233 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T22:30:58.233 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:30:58.241 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:30:58.292 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:30:58.380 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:30:58.479 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:30:58.569 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:30:58.662 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:30:58.663 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:30:58.752 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:30:58.797 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:30:58.848 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:30:58.932 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:30:58.973 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:30:58.973 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:30:59.039 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-04-24T22:30:59.046 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-04-24T22:30:59.121 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:30:59.157 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:30:59.530 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T22:30:59.540 INFO:teuthology.task.internal:Starting timer... 2024-04-24T22:30:59.540 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T22:30:59.559 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T22:30:59.575 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-24T22:30:59.576 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-04-24T22:30:59.676 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-04-24T22:30:59.681 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2024-04-24T22:30:59.733 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2024-04-24T22:30:59.735 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T22:30:59.736 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-04-24T22:30:59.808 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-04-24T22:30:59.809 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2024-04-24T22:30:59.832 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2024-04-24T22:30:59.833 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi113.front.sepia.ceph.com': 'permissive'} 2024-04-24T22:30:59.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:30:59.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:30:59.913 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:30:59.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:30:59.940 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T22:30:59.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:31:00.018 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:31:00.052 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T22:31:00.069 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T22:31:00.074 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T22:31:00.222 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T22:31:00.239 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T22:31:00.240 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi113.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T22:36:17.718 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com')] 2024-04-24T22:36:17.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-24T22:36:17.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:36:17.848 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-04-24T22:36:18.016 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-24T22:36:18.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-24T22:36:18.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:36:18.090 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-04-24T22:36:18.166 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-24T22:36:18.166 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T22:36:18.178 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T22:36:18.179 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:36:18.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:36:18.187 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T22:36:18.187 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:36:18.235 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:36:18.248 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:36:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2024-04-24T22:36:18.280 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2024-04-24T22:36:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T22:36:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:36:18.306 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:36:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T22:36:18.362 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:36:18.365 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:36:18.365 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-04-24T22:36:18.365 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:36:18.365 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:36:18.365 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:36:18.365 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:36:18.407 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-04-24T22:36:18.443 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-04-24T22:36:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T22:36:18.526 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T22:36:18.676 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:36:18.686 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:36:18.686 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-24T22:36:18.686 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:36:18.686 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:36:18.686 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:36:18.686 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:36:18.688 INFO:teuthology.run_tasks:Running task install... 2024-04-24T22:36:18.696 DEBUG:teuthology.task.install:project ceph 2024-04-24T22:36:18.696 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T22:36:18.697 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T22:36:18.697 INFO:teuthology.task.install:Using flavor: default 2024-04-24T22:36:18.702 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T22:36:18.702 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T22:36:18.703 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:36:18.703 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:36:18.705 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T22:36:18.705 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:36:18.849 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:36:18.849 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:36:18.854 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T22:36:18.854 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T22:36:19.070 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:36:19.070 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:36:19.070 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:36:19.096 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:36:19.096 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:36:19.164 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:36:19.180 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T22:36:19.180 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:36:19.180 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T22:36:19.242 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:36:19.290 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T22:36:19.290 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T22:36:19.320 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2024-04-24T22:36:19.321 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2024-04-24T22:36:19.555 INFO:teuthology.orchestra.run.smithi113.stdout:48 files removed 2024-04-24T22:36:19.582 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:36:19.641 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T22:36:19.767 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T22:36:19.851 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-04-24T22:36:19.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-04-24T22:36:20.194 INFO:teuthology.orchestra.run.smithi047.stdout:48 files removed 2024-04-24T22:36:20.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T22:36:20.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 160 kB/s | 83 kB 00:00 2024-04-24T22:36:21.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-04-24T22:36:21.508 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 102 kB/s | 83 kB 00:00 2024-04-24T22:36:21.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-04-24T22:36:22.116 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-04-24T22:36:22.801 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-04-24T22:36:23.242 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-04-24T22:36:24.824 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-04-24T22:36:26.684 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 19 MB 00:02 2024-04-24T22:36:31.518 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.2 MB 00:01 2024-04-24T22:36:32.935 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-24T22:36:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 21 MB 00:02 2024-04-24T22:36:40.487 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-24T22:36:42.507 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-04-24T22:36:42.604 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:36:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:36:42.615 INFO:teuthology.orchestra.run.smithi113.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:36:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:36:42.617 INFO:teuthology.orchestra.run.smithi113.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:36:42.617 INFO:teuthology.orchestra.run.smithi113.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:36:42.619 INFO:teuthology.orchestra.run.smithi113.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:36:42.621 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:36:42.622 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:36:42.682 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-24T22:36:42.692 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 889 k 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:36:42.693 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 221 k 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:36:42.694 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 746 k 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 298 k 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:36:42.695 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 170 k 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:36:42.696 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:36:42.697 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:36:42.698 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:36:42.699 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:36:42.700 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:36:42.701 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:36:42.702 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:36:42.703 INFO:teuthology.orchestra.run.smithi113.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:36:42.704 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:36:42.704 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:36:42.704 INFO:teuthology.orchestra.run.smithi113.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:36:42.704 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:36:42.704 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:36:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:Installing weak dependencies: 2024-04-24T22:36:42.704 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:36:42.704 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:36:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-24T22:36:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-24T22:36:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:Install 148 Packages 2024-04-24T22:36:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 4 Packages 2024-04-24T22:36:42.705 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:36:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 220 M 2024-04-24T22:36:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-24T22:36:43.263 INFO:teuthology.orchestra.run.smithi113.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-24T22:36:43.538 INFO:teuthology.orchestra.run.smithi113.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 3.2 MB/s | 889 kB 00:00 2024-04-24T22:36:43.605 INFO:teuthology.orchestra.run.smithi113.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.1 MB/s | 145 kB 00:00 2024-04-24T22:36:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 5.9 MB/s | 2.4 MB 00:00 2024-04-24T22:36:44.113 INFO:teuthology.orchestra.run.smithi113.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 5.2 MB/s | 5.4 MB 00:01 2024-04-24T22:36:44.264 INFO:teuthology.orchestra.run.smithi113.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 5.8 MB/s | 1.5 MB 00:00 2024-04-24T22:36:44.772 INFO:teuthology.orchestra.run.smithi113.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 7.1 MB/s | 4.7 MB 00:00 2024-04-24T22:36:45.589 INFO:teuthology.orchestra.run.smithi113.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 8.4 MB/s | 21 MB 00:02 2024-04-24T22:36:45.664 INFO:teuthology.orchestra.run.smithi113.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 336 kB/s | 25 kB 00:00 2024-04-24T22:36:46.148 INFO:teuthology.orchestra.run.smithi113.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 8.9 MB/s | 17 MB 00:01 2024-04-24T22:36:46.257 INFO:teuthology.orchestra.run.smithi113.stdout:(11/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 6.9 MB/s | 10 MB 00:01 2024-04-24T22:36:46.282 INFO:teuthology.orchestra.run.smithi113.stdout:(12/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 221 kB 00:00 2024-04-24T22:36:46.307 INFO:teuthology.orchestra.run.smithi113.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 542 kB/s | 27 kB 00:00 2024-04-24T22:36:46.332 INFO:teuthology.orchestra.run.smithi113.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 630 kB/s | 32 kB 00:00 2024-04-24T22:36:46.391 INFO:teuthology.orchestra.run.smithi113.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.7 MB/s | 163 kB 00:00 2024-04-24T22:36:46.441 INFO:teuthology.orchestra.run.smithi113.stdout:(16/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 5.4 MB/s | 746 kB 00:00 2024-04-24T22:36:46.466 INFO:teuthology.orchestra.run.smithi113.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.6 MB/s | 127 kB 00:00 2024-04-24T22:36:46.558 INFO:teuthology.orchestra.run.smithi113.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 4.2 MB/s | 501 kB 00:00 2024-04-24T22:36:46.626 INFO:teuthology.orchestra.run.smithi113.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 676 kB/s | 45 kB 00:00 2024-04-24T22:36:46.701 INFO:teuthology.orchestra.run.smithi113.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.7 MB/s | 133 kB 00:00 2024-04-24T22:36:46.776 INFO:teuthology.orchestra.run.smithi113.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.1 MB/s | 162 kB 00:00 2024-04-24T22:36:46.885 INFO:teuthology.orchestra.run.smithi113.stdout:(22/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 13 MB/s | 5.3 MB 00:00 2024-04-24T22:36:46.910 INFO:teuthology.orchestra.run.smithi113.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 321 kB 00:00 2024-04-24T22:36:46.952 INFO:teuthology.orchestra.run.smithi113.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 4.4 MB/s | 298 kB 00:00 2024-04-24T22:36:46.977 INFO:teuthology.orchestra.run.smithi113.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 99 kB 00:00 2024-04-24T22:36:47.011 INFO:teuthology.orchestra.run.smithi113.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-24T22:36:47.077 INFO:teuthology.orchestra.run.smithi113.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-24T22:36:47.145 INFO:teuthology.orchestra.run.smithi113.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 476 kB/s | 32 kB 00:00 2024-04-24T22:36:47.212 INFO:teuthology.orchestra.run.smithi113.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.1 MB/s | 141 kB 00:00 2024-04-24T22:36:47.354 INFO:teuthology.orchestra.run.smithi113.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 8.2 MB/s | 3.1 MB 00:00 2024-04-24T22:36:47.545 INFO:teuthology.orchestra.run.smithi113.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 11 MB/s | 3.5 MB 00:00 2024-04-24T22:36:47.629 INFO:teuthology.orchestra.run.smithi113.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.9 MB/s | 249 kB 00:00 2024-04-24T22:36:47.696 INFO:teuthology.orchestra.run.smithi113.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 740 kB/s | 49 kB 00:00 2024-04-24T22:36:47.755 INFO:teuthology.orchestra.run.smithi113.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 263 kB/s | 15 kB 00:00 2024-04-24T22:36:47.822 INFO:teuthology.orchestra.run.smithi113.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.9 MB/s | 264 kB 00:00 2024-04-24T22:36:47.930 INFO:teuthology.orchestra.run.smithi113.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-24T22:36:47.981 INFO:teuthology.orchestra.run.smithi113.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 365 kB/s | 18 kB 00:00 2024-04-24T22:36:48.049 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.2 MB 00:01 2024-04-24T22:36:48.090 INFO:teuthology.orchestra.run.smithi113.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 741 kB/s | 80 kB 00:00 2024-04-24T22:36:48.240 INFO:teuthology.orchestra.run.smithi113.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 8.3 MB/s | 7.4 MB 00:00 2024-04-24T22:36:48.273 INFO:teuthology.orchestra.run.smithi113.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-24T22:36:48.315 INFO:teuthology.orchestra.run.smithi113.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-24T22:36:48.382 INFO:teuthology.orchestra.run.smithi113.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-24T22:36:48.410 INFO:teuthology.orchestra.run.smithi113.stdout:(43/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-24T22:36:48.433 INFO:teuthology.orchestra.run.smithi113.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-24T22:36:48.458 INFO:teuthology.orchestra.run.smithi113.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-24T22:36:48.484 INFO:teuthology.orchestra.run.smithi113.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-24T22:36:48.509 INFO:teuthology.orchestra.run.smithi113.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-24T22:36:48.551 INFO:teuthology.orchestra.run.smithi113.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-04-24T22:36:48.593 INFO:teuthology.orchestra.run.smithi113.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-24T22:36:48.626 INFO:teuthology.orchestra.run.smithi113.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-04-24T22:36:48.652 INFO:teuthology.orchestra.run.smithi113.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 368 kB/s | 21 kB 00:00 2024-04-24T22:36:48.677 INFO:teuthology.orchestra.run.smithi113.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-24T22:36:48.702 INFO:teuthology.orchestra.run.smithi113.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-04-24T22:36:48.752 INFO:teuthology.orchestra.run.smithi113.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-24T22:36:48.803 INFO:teuthology.orchestra.run.smithi113.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-24T22:36:48.828 INFO:teuthology.orchestra.run.smithi113.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-04-24T22:36:48.853 INFO:teuthology.orchestra.run.smithi113.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-24T22:36:48.879 INFO:teuthology.orchestra.run.smithi113.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-24T22:36:48.912 INFO:teuthology.orchestra.run.smithi113.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-24T22:36:48.946 INFO:teuthology.orchestra.run.smithi113.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-24T22:36:48.971 INFO:teuthology.orchestra.run.smithi113.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-24T22:36:48.988 INFO:teuthology.orchestra.run.smithi113.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-24T22:36:49.022 INFO:teuthology.orchestra.run.smithi113.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-24T22:36:49.047 INFO:teuthology.orchestra.run.smithi113.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-24T22:36:49.064 INFO:teuthology.orchestra.run.smithi113.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 848 kB/s | 35 kB 00:00 2024-04-24T22:36:49.156 INFO:teuthology.orchestra.run.smithi113.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 44 MB/s | 4.7 MB 00:00 2024-04-24T22:36:49.240 INFO:teuthology.orchestra.run.smithi113.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-04-24T22:36:49.282 INFO:teuthology.orchestra.run.smithi113.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-24T22:36:49.315 INFO:teuthology.orchestra.run.smithi113.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-24T22:36:49.340 INFO:teuthology.orchestra.run.smithi113.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-24T22:36:49.774 INFO:teuthology.orchestra.run.smithi113.stdout:(71/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 12 MB/s | 48 MB 00:04 2024-04-24T22:36:49.816 INFO:teuthology.orchestra.run.smithi113.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.1 MB/s | 1.0 MB 00:00 2024-04-24T22:36:49.908 INFO:teuthology.orchestra.run.smithi113.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 9.8 MB/s | 5.8 MB 00:00 2024-04-24T22:36:49.933 INFO:teuthology.orchestra.run.smithi113.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 552 kB/s | 87 kB 00:00 2024-04-24T22:36:49.959 INFO:teuthology.orchestra.run.smithi113.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-24T22:36:49.999 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-24T22:36:50.000 INFO:teuthology.orchestra.run.smithi113.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-04-24T22:36:50.026 INFO:teuthology.orchestra.run.smithi113.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-24T22:36:50.051 INFO:teuthology.orchestra.run.smithi113.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-24T22:36:50.076 INFO:teuthology.orchestra.run.smithi113.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-24T22:36:50.102 INFO:teuthology.orchestra.run.smithi113.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-04-24T22:36:50.177 INFO:teuthology.orchestra.run.smithi113.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-24T22:36:50.219 INFO:teuthology.orchestra.run.smithi113.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-04-24T22:36:50.244 INFO:teuthology.orchestra.run.smithi113.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-24T22:36:50.269 INFO:teuthology.orchestra.run.smithi113.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T22:36:50.295 INFO:teuthology.orchestra.run.smithi113.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-04-24T22:36:50.320 INFO:teuthology.orchestra.run.smithi113.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-24T22:36:50.345 INFO:teuthology.orchestra.run.smithi113.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-24T22:36:50.405 INFO:teuthology.orchestra.run.smithi113.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-24T22:36:50.405 INFO:teuthology.orchestra.run.smithi113.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T22:36:50.421 INFO:teuthology.orchestra.run.smithi113.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T22:36:50.464 INFO:teuthology.orchestra.run.smithi113.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 998 kB/s | 42 kB 00:00 2024-04-24T22:36:50.539 INFO:teuthology.orchestra.run.smithi113.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-24T22:36:50.564 INFO:teuthology.orchestra.run.smithi113.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 442 kB/s | 44 kB 00:00 2024-04-24T22:36:50.589 INFO:teuthology.orchestra.run.smithi113.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-24T22:36:50.614 INFO:teuthology.orchestra.run.smithi113.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-04-24T22:36:50.640 INFO:teuthology.orchestra.run.smithi113.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-04-24T22:36:50.667 INFO:teuthology.orchestra.run.smithi113.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-24T22:36:50.682 INFO:teuthology.orchestra.run.smithi113.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-04-24T22:36:50.707 INFO:teuthology.orchestra.run.smithi113.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-24T22:36:50.724 INFO:teuthology.orchestra.run.smithi113.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-24T22:36:50.741 INFO:teuthology.orchestra.run.smithi113.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-24T22:36:50.950 INFO:teuthology.orchestra.run.smithi113.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-04-24T22:36:50.966 INFO:teuthology.orchestra.run.smithi113.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-04-24T22:36:51.200 INFO:teuthology.orchestra.run.smithi113.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-04-24T22:36:51.225 INFO:teuthology.orchestra.run.smithi113.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-24T22:36:51.250 INFO:teuthology.orchestra.run.smithi113.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 391 kB/s | 19 kB 00:00 2024-04-24T22:36:51.276 INFO:teuthology.orchestra.run.smithi113.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-24T22:36:51.426 INFO:teuthology.orchestra.run.smithi113.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-24T22:36:51.451 INFO:teuthology.orchestra.run.smithi113.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-04-24T22:36:51.477 INFO:teuthology.orchestra.run.smithi113.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 244 kB/s | 49 kB 00:00 2024-04-24T22:36:51.502 INFO:teuthology.orchestra.run.smithi113.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 898 kB/s | 67 kB 00:00 2024-04-24T22:36:51.527 INFO:teuthology.orchestra.run.smithi113.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-24T22:36:51.576 INFO:teuthology.orchestra.run.smithi113.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.3 MB/s | 838 kB 00:00 2024-04-24T22:36:51.603 INFO:teuthology.orchestra.run.smithi113.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-24T22:36:51.628 INFO:teuthology.orchestra.run.smithi113.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 443 kB/s | 43 kB 00:00 2024-04-24T22:36:51.653 INFO:teuthology.orchestra.run.smithi113.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 419 kB/s | 32 kB 00:00 2024-04-24T22:36:51.678 INFO:teuthology.orchestra.run.smithi113.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 301 kB/s | 15 kB 00:00 2024-04-24T22:36:51.704 INFO:teuthology.orchestra.run.smithi113.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-24T22:36:51.729 INFO:teuthology.orchestra.run.smithi113.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-24T22:36:51.754 INFO:teuthology.orchestra.run.smithi113.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-04-24T22:36:51.779 INFO:teuthology.orchestra.run.smithi113.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 474 kB/s | 35 kB 00:00 2024-04-24T22:36:51.804 INFO:teuthology.orchestra.run.smithi113.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-24T22:36:51.880 INFO:teuthology.orchestra.run.smithi113.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-04-24T22:36:51.905 INFO:teuthology.orchestra.run.smithi113.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-24T22:36:51.930 INFO:teuthology.orchestra.run.smithi113.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 86 kB/s | 11 kB 00:00 2024-04-24T22:36:51.955 INFO:teuthology.orchestra.run.smithi113.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2024-04-24T22:36:51.989 INFO:teuthology.orchestra.run.smithi113.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-04-24T22:36:52.015 INFO:teuthology.orchestra.run.smithi113.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-24T22:36:52.040 INFO:teuthology.orchestra.run.smithi113.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-24T22:36:52.082 INFO:teuthology.orchestra.run.smithi113.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-04-24T22:36:52.107 INFO:teuthology.orchestra.run.smithi113.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 505 kB/s | 46 kB 00:00 2024-04-24T22:36:52.124 INFO:teuthology.orchestra.run.smithi113.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 941 kB/s | 79 kB 00:00 2024-04-24T22:36:52.149 INFO:teuthology.orchestra.run.smithi113.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 866 kB/s | 58 kB 00:00 2024-04-24T22:36:52.183 INFO:teuthology.orchestra.run.smithi113.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-24T22:36:52.208 INFO:teuthology.orchestra.run.smithi113.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-04-24T22:36:52.233 INFO:teuthology.orchestra.run.smithi113.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-24T22:36:52.258 INFO:teuthology.orchestra.run.smithi113.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 567 kB/s | 28 kB 00:00 2024-04-24T22:36:52.283 INFO:teuthology.orchestra.run.smithi113.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 410 kB/s | 31 kB 00:00 2024-04-24T22:36:52.309 INFO:teuthology.orchestra.run.smithi113.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-24T22:36:52.334 INFO:teuthology.orchestra.run.smithi113.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-24T22:36:52.359 INFO:teuthology.orchestra.run.smithi113.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-04-24T22:36:52.397 INFO:teuthology.orchestra.run.smithi113.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 580 kB/s | 51 kB 00:00 2024-04-24T22:36:52.418 INFO:teuthology.orchestra.run.smithi113.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-24T22:36:52.443 INFO:teuthology.orchestra.run.smithi113.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-24T22:36:52.469 INFO:teuthology.orchestra.run.smithi113.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.9 MB/s | 427 kB 00:00 2024-04-24T22:36:52.494 INFO:teuthology.orchestra.run.smithi113.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-04-24T22:36:52.510 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 21 MB 00:02 2024-04-24T22:36:52.519 INFO:teuthology.orchestra.run.smithi113.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-24T22:36:52.561 INFO:teuthology.orchestra.run.smithi113.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-24T22:36:52.720 INFO:teuthology.orchestra.run.smithi113.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 1.3 MB 00:00 2024-04-24T22:36:52.804 INFO:teuthology.orchestra.run.smithi113.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-04-24T22:36:52.862 INFO:teuthology.orchestra.run.smithi113.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 9.2 MB/s | 3.4 MB 00:00 2024-04-24T22:36:52.971 INFO:teuthology.orchestra.run.smithi113.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-04-24T22:36:52.976 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:36:52.976 INFO:teuthology.orchestra.run.smithi113.stdout:Total 21 MB/s | 220 MB 00:10 2024-04-24T22:36:54.099 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-24T22:36:54.241 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-24T22:36:54.241 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-24T22:36:55.990 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-24T22:36:55.991 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-24T22:36:58.643 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-24T22:37:01.113 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:37:01.116 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T22:37:01.137 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T22:37:01.140 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T22:37:01.142 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T22:37:01.142 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T22:37:01.144 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T22:37:01.159 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T22:37:01.161 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T22:37:01.224 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-24T22:37:01.259 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-24T22:37:01.260 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-24T22:37:01.260 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-24T22:37:01.260 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-24T22:37:01.260 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T22:37:01.260 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T22:37:01.260 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T22:37:01.260 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 889 k 2024-04-24T22:37:01.261 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T22:37:01.261 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T22:37:01.261 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T22:37:01.261 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T22:37:01.261 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T22:37:01.261 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T22:37:01.261 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T22:37:01.261 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T22:37:01.261 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T22:37:01.261 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 221 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 746 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T22:37:01.262 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 298 k 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 170 k 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T22:37:01.263 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T22:37:01.264 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T22:37:01.265 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T22:37:01.266 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T22:37:01.267 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T22:37:01.267 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T22:37:01.267 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T22:37:01.267 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T22:37:01.267 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T22:37:01.267 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T22:37:01.267 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T22:37:01.267 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T22:37:01.267 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T22:37:01.267 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T22:37:01.268 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T22:37:01.268 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T22:37:01.268 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T22:37:01.268 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T22:37:01.268 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T22:37:01.268 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T22:37:01.268 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T22:37:01.269 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T22:37:01.269 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T22:37:01.269 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T22:37:01.269 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T22:37:01.269 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T22:37:01.269 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T22:37:01.270 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T22:37:01.270 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T22:37:01.270 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T22:37:01.270 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T22:37:01.270 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T22:37:01.270 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T22:37:01.271 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T22:37:01.271 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T22:37:01.271 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T22:37:01.271 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T22:37:01.271 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T22:37:01.271 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T22:37:01.271 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T22:37:01.272 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T22:37:01.272 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T22:37:01.272 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T22:37:01.272 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T22:37:01.272 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T22:37:01.272 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T22:37:01.272 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T22:37:01.273 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T22:37:01.273 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T22:37:01.273 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T22:37:01.273 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T22:37:01.273 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T22:37:01.273 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T22:37:01.273 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T22:37:01.274 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T22:37:01.274 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T22:37:01.274 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T22:37:01.274 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T22:37:01.274 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T22:37:01.274 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T22:37:01.274 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T22:37:01.275 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T22:37:01.275 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T22:37:01.275 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T22:37:01.275 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T22:37:01.275 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T22:37:01.275 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T22:37:01.275 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T22:37:01.276 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T22:37:01.276 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T22:37:01.276 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T22:37:01.276 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T22:37:01.276 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T22:37:01.276 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T22:37:01.276 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T22:37:01.277 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T22:37:01.277 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T22:37:01.277 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T22:37:01.277 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-04-24T22:37:01.277 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T22:37:01.277 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:37:01.277 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-24T22:37:01.278 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-24T22:37:01.278 INFO:teuthology.orchestra.run.smithi047.stdout:Install 148 Packages 2024-04-24T22:37:01.278 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 4 Packages 2024-04-24T22:37:01.278 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:37:01.279 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 220 M 2024-04-24T22:37:01.279 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-24T22:37:02.041 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-24T22:37:02.132 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T22:37:02.179 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T22:37:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T22:37:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T22:37:02.325 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T22:37:02.368 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T22:37:02.545 INFO:teuthology.orchestra.run.smithi047.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-24T22:37:02.710 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T22:37:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:37:02.870 INFO:teuthology.orchestra.run.smithi047.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 2.7 MB/s | 889 kB 00:00 2024-04-24T22:37:02.948 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:37:02.962 INFO:teuthology.orchestra.run.smithi047.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.6 MB/s | 145 kB 00:00 2024-04-24T22:37:02.986 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:37:03.062 INFO:teuthology.orchestra.run.smithi047.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 6.8 MB/s | 5.4 MB 00:00 2024-04-24T22:37:03.077 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:37:03.106 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:37:03.263 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T22:37:03.312 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T22:37:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:37:03.436 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:37:03.454 INFO:teuthology.orchestra.run.smithi047.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 4.8 MB/s | 2.4 MB 00:00 2024-04-24T22:37:03.466 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:37:03.512 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:37:03.521 INFO:teuthology.orchestra.run.smithi047.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-04-24T22:37:03.559 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:37:03.581 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:37:03.817 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:37:03.850 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:37:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T22:37:04.043 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T22:37:04.082 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T22:37:04.117 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T22:37:04.146 INFO:teuthology.orchestra.run.smithi047.stdout:(7/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 11 MB/s | 21 MB 00:01 2024-04-24T22:37:04.182 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T22:37:04.263 INFO:teuthology.orchestra.run.smithi047.stdout:(8/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 5.8 MB/s | 4.7 MB 00:00 2024-04-24T22:37:04.347 INFO:teuthology.orchestra.run.smithi047.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 306 kB/s | 25 kB 00:00 2024-04-24T22:37:04.383 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T22:37:04.428 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T22:37:04.460 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T22:37:04.496 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T22:37:04.566 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T22:37:04.619 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T22:37:04.657 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T22:37:04.700 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T22:37:04.775 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T22:37:04.812 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T22:37:04.855 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T22:37:04.883 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T22:37:04.952 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T22:37:05.006 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T22:37:05.014 INFO:teuthology.orchestra.run.smithi047.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 12 MB/s | 10 MB 00:00 2024-04-24T22:37:05.079 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T22:37:05.106 INFO:teuthology.orchestra.run.smithi047.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 221 kB 00:00 2024-04-24T22:37:05.181 INFO:teuthology.orchestra.run.smithi047.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 365 kB/s | 27 kB 00:00 2024-04-24T22:37:05.214 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T22:37:05.239 INFO:teuthology.orchestra.run.smithi047.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 555 kB/s | 32 kB 00:00 2024-04-24T22:37:05.266 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T22:37:05.357 INFO:teuthology.orchestra.run.smithi047.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 6.3 MB/s | 746 kB 00:00 2024-04-24T22:37:05.607 INFO:teuthology.orchestra.run.smithi047.stdout:(15/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 8.1 MB/s | 17 MB 00:02 2024-04-24T22:37:05.640 INFO:teuthology.orchestra.run.smithi047.stdout:(16/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 578 kB/s | 163 kB 00:00 2024-04-24T22:37:05.674 INFO:teuthology.orchestra.run.smithi047.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.9 MB/s | 127 kB 00:00 2024-04-24T22:37:05.724 INFO:teuthology.orchestra.run.smithi047.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.9 MB/s | 501 kB 00:00 2024-04-24T22:37:05.792 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T22:37:05.799 INFO:teuthology.orchestra.run.smithi047.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 611 kB/s | 45 kB 00:00 2024-04-24T22:37:05.875 INFO:teuthology.orchestra.run.smithi047.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.8 MB/s | 133 kB 00:00 2024-04-24T22:37:05.880 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T22:37:05.909 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T22:37:05.951 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T22:37:05.976 INFO:teuthology.orchestra.run.smithi047.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.6 MB/s | 162 kB 00:00 2024-04-24T22:37:05.990 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T22:37:06.032 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T22:37:06.050 INFO:teuthology.orchestra.run.smithi047.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 4.2 MB/s | 321 kB 00:00 2024-04-24T22:37:06.134 INFO:teuthology.orchestra.run.smithi047.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.5 MB/s | 298 kB 00:00 2024-04-24T22:37:06.196 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T22:37:06.259 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T22:37:06.276 INFO:teuthology.orchestra.run.smithi047.stdout:(24/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 8.9 MB/s | 5.3 MB 00:00 2024-04-24T22:37:06.310 INFO:teuthology.orchestra.run.smithi047.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 571 kB/s | 99 kB 00:00 2024-04-24T22:37:06.314 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T22:37:06.335 INFO:teuthology.orchestra.run.smithi047.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.5 MB/s | 85 kB 00:00 2024-04-24T22:37:06.387 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T22:37:06.419 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T22:37:06.419 INFO:teuthology.orchestra.run.smithi047.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-24T22:37:06.454 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T22:37:06.494 INFO:teuthology.orchestra.run.smithi047.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 428 kB/s | 32 kB 00:00 2024-04-24T22:37:06.509 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T22:37:06.546 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T22:37:06.580 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T22:37:06.611 INFO:teuthology.orchestra.run.smithi047.stdout:(29/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 10 MB/s | 3.1 MB 00:00 2024-04-24T22:37:06.623 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T22:37:06.645 INFO:teuthology.orchestra.run.smithi047.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 947 kB/s | 141 kB 00:00 2024-04-24T22:37:06.758 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T22:37:06.803 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T22:37:06.834 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T22:37:06.874 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T22:37:06.969 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T22:37:06.986 INFO:teuthology.orchestra.run.smithi047.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 9.4 MB/s | 3.5 MB 00:00 2024-04-24T22:37:07.000 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T22:37:07.031 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T22:37:07.053 INFO:teuthology.orchestra.run.smithi047.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.7 MB/s | 249 kB 00:00 2024-04-24T22:37:07.091 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T22:37:07.115 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T22:37:07.120 INFO:teuthology.orchestra.run.smithi047.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 749 kB/s | 49 kB 00:00 2024-04-24T22:37:07.190 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T22:37:07.195 INFO:teuthology.orchestra.run.smithi047.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 207 kB/s | 15 kB 00:00 2024-04-24T22:37:07.271 INFO:teuthology.orchestra.run.smithi047.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.5 MB/s | 264 kB 00:00 2024-04-24T22:37:07.318 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T22:37:07.379 INFO:teuthology.orchestra.run.smithi047.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-24T22:37:07.446 INFO:teuthology.orchestra.run.smithi047.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 278 kB/s | 18 kB 00:00 2024-04-24T22:37:07.515 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T22:37:07.543 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T22:37:07.577 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T22:37:07.596 INFO:teuthology.orchestra.run.smithi047.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 7.8 MB/s | 7.4 MB 00:00 2024-04-24T22:37:07.616 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T22:37:07.657 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T22:37:07.686 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T22:37:07.720 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T22:37:07.754 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T22:37:07.796 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T22:37:07.830 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T22:37:07.889 INFO:teuthology.orchestra.run.smithi047.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 182 kB/s | 80 kB 00:00 2024-04-24T22:37:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T22:37:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T22:37:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T22:37:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T22:37:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T22:37:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T22:37:08.564 INFO:teuthology.orchestra.run.smithi047.stdout:(40/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 11 MB/s | 48 MB 00:04 2024-04-24T22:37:08.631 INFO:teuthology.orchestra.run.smithi047.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 837 kB/s | 803 kB 00:00 2024-04-24T22:37:08.739 INFO:teuthology.orchestra.run.smithi047.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 230 kB/s | 195 kB 00:00 2024-04-24T22:37:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T22:37:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T22:37:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T22:37:09.058 INFO:teuthology.orchestra.run.smithi047.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 514 kB/s | 253 kB 00:00 2024-04-24T22:37:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T22:37:09.157 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T22:37:09.200 INFO:teuthology.orchestra.run.smithi047.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 266 kB/s | 106 kB 00:00 2024-04-24T22:37:09.242 INFO:teuthology.orchestra.run.smithi047.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 2.3 MB/s | 1.2 MB 00:00 2024-04-24T22:37:09.243 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T22:37:09.278 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T22:37:09.452 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T22:37:09.482 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T22:37:09.501 INFO:teuthology.orchestra.run.smithi047.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 307 kB/s | 135 kB 00:00 2024-04-24T22:37:09.560 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T22:37:09.567 INFO:teuthology.orchestra.run.smithi047.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 341 kB/s | 125 kB 00:00 2024-04-24T22:37:09.652 INFO:teuthology.orchestra.run.smithi047.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 528 kB/s | 215 kB 00:00 2024-04-24T22:37:09.860 INFO:teuthology.orchestra.run.smithi047.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 517 kB/s | 107 kB 00:00 2024-04-24T22:37:09.911 INFO:teuthology.orchestra.run.smithi047.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-04-24T22:37:09.951 INFO:teuthology.orchestra.run.smithi047.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 689 kB 00:00 2024-04-24T22:37:10.027 INFO:teuthology.orchestra.run.smithi047.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 129 kB/s | 21 kB 00:00 2024-04-24T22:37:10.111 INFO:teuthology.orchestra.run.smithi047.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 207 kB/s | 33 kB 00:00 2024-04-24T22:37:10.186 INFO:teuthology.orchestra.run.smithi047.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-04-24T22:37:10.201 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T22:37:10.253 INFO:teuthology.orchestra.run.smithi047.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 129 kB/s | 18 kB 00:00 2024-04-24T22:37:10.313 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T22:37:10.320 INFO:teuthology.orchestra.run.smithi047.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 84 kB/s | 11 kB 00:00 2024-04-24T22:37:10.462 INFO:teuthology.orchestra.run.smithi047.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 772 kB/s | 160 kB 00:00 2024-04-24T22:37:10.539 INFO:teuthology.orchestra.run.smithi047.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 207 kB/s | 45 kB 00:00 2024-04-24T22:37:10.729 INFO:teuthology.orchestra.run.smithi047.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.3 MB/s | 3.0 MB 00:00 2024-04-24T22:37:10.888 INFO:teuthology.orchestra.run.smithi047.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.5 MB/s | 662 kB 00:00 2024-04-24T22:37:10.933 INFO:teuthology.orchestra.run.smithi047.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 126 kB/s | 25 kB 00:00 2024-04-24T22:37:10.971 INFO:teuthology.orchestra.run.smithi047.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 1.3 MB/s | 582 kB 00:00 2024-04-24T22:37:11.088 INFO:teuthology.orchestra.run.smithi047.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 238 kB/s | 47 kB 00:00 2024-04-24T22:37:11.230 INFO:teuthology.orchestra.run.smithi047.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 731 kB/s | 188 kB 00:00 2024-04-24T22:37:11.264 INFO:teuthology.orchestra.run.smithi047.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 888 kB/s | 292 kB 00:00 2024-04-24T22:37:11.289 INFO:teuthology.orchestra.run.smithi047.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 176 kB/s | 35 kB 00:00 2024-04-24T22:37:11.534 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T22:37:11.620 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T22:37:11.647 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T22:37:11.681 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T22:37:11.814 INFO:teuthology.orchestra.run.smithi047.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-04-24T22:37:11.992 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T22:37:12.035 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:37:12.064 INFO:teuthology.orchestra.run.smithi047.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-04-24T22:37:12.134 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:37:12.165 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T22:37:12.201 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T22:37:12.265 INFO:teuthology.orchestra.run.smithi047.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 2.0 MB/s | 911 kB 00:00 2024-04-24T22:37:12.290 INFO:teuthology.orchestra.run.smithi047.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 596 kB/s | 133 kB 00:00 2024-04-24T22:37:12.657 INFO:teuthology.orchestra.run.smithi047.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-04-24T22:37:12.672 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T22:37:12.696 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:37:12.760 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:37:12.790 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T22:37:12.849 INFO:teuthology.orchestra.run.smithi047.stdout:(72/152): openblas-openmp-0.3.21-2.el9.x86_64.r 2.9 MB/s | 4.7 MB 00:01 2024-04-24T22:37:12.903 INFO:teuthology.orchestra.run.smithi047.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 357 kB/s | 87 kB 00:00 2024-04-24T22:37:13.166 INFO:teuthology.orchestra.run.smithi047.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 932 kB/s | 244 kB 00:00 2024-04-24T22:37:13.408 INFO:teuthology.orchestra.run.smithi047.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 5.1 MB/s | 5.8 MB 00:01 2024-04-24T22:37:13.458 INFO:teuthology.orchestra.run.smithi047.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 852 kB/s | 248 kB 00:00 2024-04-24T22:37:13.584 INFO:teuthology.orchestra.run.smithi047.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 147 kB/s | 26 kB 00:00 2024-04-24T22:37:13.626 INFO:teuthology.orchestra.run.smithi047.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 115 kB/s | 19 kB 00:00 2024-04-24T22:37:13.784 INFO:teuthology.orchestra.run.smithi047.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 220 kB/s | 35 kB 00:00 2024-04-24T22:37:13.860 INFO:teuthology.orchestra.run.smithi047.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 627 kB/s | 172 kB 00:00 2024-04-24T22:37:13.985 INFO:teuthology.orchestra.run.smithi047.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 5.3 MB/s | 6.0 MB 00:01 2024-04-24T22:37:14.210 INFO:teuthology.orchestra.run.smithi047.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 717 kB/s | 250 kB 00:00 2024-04-24T22:37:14.277 INFO:teuthology.orchestra.run.smithi047.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 764 kB/s | 222 kB 00:00 2024-04-24T22:37:14.419 INFO:teuthology.orchestra.run.smithi047.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 372 kB/s | 77 kB 00:00 2024-04-24T22:37:14.511 INFO:teuthology.orchestra.run.smithi047.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 182 kB/s | 42 kB 00:00 2024-04-24T22:37:14.778 INFO:teuthology.orchestra.run.smithi047.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 753 kB/s | 269 kB 00:00 2024-04-24T22:37:14.803 INFO:teuthology.orchestra.run.smithi047.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 545 kB/s | 159 kB 00:00 2024-04-24T22:37:14.978 INFO:teuthology.orchestra.run.smithi047.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-04-24T22:37:15.120 INFO:teuthology.orchestra.run.smithi047.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 161 kB/s | 51 kB 00:00 2024-04-24T22:37:15.154 INFO:teuthology.orchestra.run.smithi047.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 744 kB/s | 279 kB 00:00 2024-04-24T22:37:15.196 INFO:teuthology.orchestra.run.smithi047.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 248 kB/s | 54 kB 00:00 2024-04-24T22:37:15.212 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:37:15.245 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:37:15.404 INFO:teuthology.orchestra.run.smithi047.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 167 kB/s | 42 kB 00:00 2024-04-24T22:37:15.454 INFO:teuthology.orchestra.run.smithi047.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 171 kB/s | 44 kB 00:00 2024-04-24T22:37:15.630 INFO:teuthology.orchestra.run.smithi047.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 206 kB/s | 36 kB 00:00 2024-04-24T22:37:15.805 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:37:15.838 INFO:teuthology.orchestra.run.smithi047.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 510 kB/s | 221 kB 00:00 2024-04-24T22:37:15.857 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:37:15.948 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:37:15.948 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T22:37:15.948 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T22:37:15.948 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:15.955 INFO:teuthology.orchestra.run.smithi047.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 146 kB/s | 47 kB 00:00 2024-04-24T22:37:15.983 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:37:16.181 INFO:teuthology.orchestra.run.smithi047.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 258 kB/s | 58 kB 00:00 2024-04-24T22:37:16.214 INFO:teuthology.orchestra.run.smithi047.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 813 kB/s | 305 kB 00:00 2024-04-24T22:37:16.373 INFO:teuthology.orchestra.run.smithi047.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 491 kB/s | 94 kB 00:00 2024-04-24T22:37:16.415 INFO:teuthology.orchestra.run.smithi047.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 318 kB/s | 64 kB 00:00 2024-04-24T22:37:16.532 INFO:teuthology.orchestra.run.smithi047.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 193 kB/s | 22 kB 00:00 2024-04-24T22:37:16.599 INFO:teuthology.orchestra.run.smithi047.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 199 kB/s | 45 kB 00:00 2024-04-24T22:37:16.766 INFO:teuthology.orchestra.run.smithi047.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-04-24T22:37:17.250 INFO:teuthology.orchestra.run.smithi047.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 844 kB/s | 548 kB 00:00 2024-04-24T22:37:17.363 INFO:teuthology.orchestra.run.smithi047.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 173 kB/s | 19 kB 00:00 2024-04-24T22:37:17.576 INFO:teuthology.orchestra.run.smithi047.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 382 kB/s | 309 kB 00:00 2024-04-24T22:37:17.701 INFO:teuthology.orchestra.run.smithi047.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 209 kB/s | 26 kB 00:00 2024-04-24T22:37:17.826 INFO:teuthology.orchestra.run.smithi047.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 392 kB/s | 49 kB 00:00 2024-04-24T22:37:17.910 INFO:teuthology.orchestra.run.smithi047.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-04-24T22:37:17.943 INFO:teuthology.orchestra.run.smithi047.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 581 kB/s | 67 kB 00:00 2024-04-24T22:37:18.035 INFO:teuthology.orchestra.run.smithi047.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-24T22:37:18.177 INFO:teuthology.orchestra.run.smithi047.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-04-24T22:37:18.253 INFO:teuthology.orchestra.run.smithi047.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-24T22:37:18.286 INFO:teuthology.orchestra.run.smithi047.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-04-24T22:37:18.370 INFO:teuthology.orchestra.run.smithi047.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2024-04-24T22:37:18.420 INFO:teuthology.orchestra.run.smithi047.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 113 kB/s | 15 kB 00:00 2024-04-24T22:37:18.487 INFO:teuthology.orchestra.run.smithi047.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-24T22:37:18.554 INFO:teuthology.orchestra.run.smithi047.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-24T22:37:18.612 INFO:teuthology.orchestra.run.smithi047.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-24T22:37:18.663 INFO:teuthology.orchestra.run.smithi047.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 327 kB/s | 35 kB 00:00 2024-04-24T22:37:18.721 INFO:teuthology.orchestra.run.smithi047.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-04-24T22:37:18.847 INFO:teuthology.orchestra.run.smithi047.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-24T22:37:18.964 INFO:teuthology.orchestra.run.smithi047.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-24T22:37:19.031 INFO:teuthology.orchestra.run.smithi047.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.6 MB/s | 2.0 MB 00:00 2024-04-24T22:37:19.072 INFO:teuthology.orchestra.run.smithi047.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-04-24T22:37:19.131 INFO:teuthology.orchestra.run.smithi047.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-04-24T22:37:19.173 INFO:teuthology.orchestra.run.smithi047.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-04-24T22:37:19.240 INFO:teuthology.orchestra.run.smithi047.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-04-24T22:37:19.324 INFO:teuthology.orchestra.run.smithi047.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-04-24T22:37:19.349 INFO:teuthology.orchestra.run.smithi047.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2024-04-24T22:37:19.501 INFO:teuthology.orchestra.run.smithi047.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 447 kB/s | 79 kB 00:00 2024-04-24T22:37:19.566 INFO:teuthology.orchestra.run.smithi047.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 266 kB/s | 58 kB 00:00 2024-04-24T22:37:19.633 INFO:teuthology.orchestra.run.smithi047.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-24T22:37:19.684 INFO:teuthology.orchestra.run.smithi047.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-04-24T22:37:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 832 kB/s | 90 kB 00:00 2024-04-24T22:37:19.792 INFO:teuthology.orchestra.run.smithi047.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 261 kB/s | 28 kB 00:00 2024-04-24T22:37:19.859 INFO:teuthology.orchestra.run.smithi047.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 265 kB/s | 31 kB 00:00 2024-04-24T22:37:19.918 INFO:teuthology.orchestra.run.smithi047.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-24T22:37:19.995 INFO:teuthology.orchestra.run.smithi047.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 437 kB/s | 59 kB 00:00 2024-04-24T22:37:20.035 INFO:teuthology.orchestra.run.smithi047.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 307 kB/s | 36 kB 00:00 2024-04-24T22:37:20.119 INFO:teuthology.orchestra.run.smithi047.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 416 kB/s | 51 kB 00:00 2024-04-24T22:37:20.161 INFO:teuthology.orchestra.run.smithi047.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-24T22:37:20.236 INFO:teuthology.orchestra.run.smithi047.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 772 kB/s | 90 kB 00:00 2024-04-24T22:37:20.295 INFO:teuthology.orchestra.run.smithi047.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-24T22:37:20.345 INFO:teuthology.orchestra.run.smithi047.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-04-24T22:37:20.429 INFO:teuthology.orchestra.run.smithi047.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-24T22:37:20.670 INFO:teuthology.orchestra.run.smithi047.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-04-24T22:37:20.813 INFO:teuthology.orchestra.run.smithi047.stdout:(148/152): librados2-19.0.0-2445.gd2c45d97.el9. 8.8 MB/s | 3.4 MB 00:00 2024-04-24T22:37:21.038 INFO:teuthology.orchestra.run.smithi047.stdout:(149/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 8.6 MB/s | 3.1 MB 00:00 2024-04-24T22:37:21.621 INFO:teuthology.orchestra.run.smithi047.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 1.3 MB 00:00 2024-04-24T22:37:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:(151/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.2 MB/s | 16 MB 00:07 2024-04-24T22:37:22.605 INFO:teuthology.orchestra.run.smithi047.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 2.4 MB 00:01 2024-04-24T22:37:22.618 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:37:22.619 INFO:teuthology.orchestra.run.smithi047.stdout:Total 10 MB/s | 220 MB 00:21 2024-04-24T22:37:25.637 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-24T22:37:25.894 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-24T22:37:25.894 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-24T22:37:27.546 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:37:27.546 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2024-04-24T22:37:27.546 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2024-04-24T22:37:27.546 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2024-04-24T22:37:27.546 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2024-04-24T22:37:27.546 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2024-04-24T22:37:27.547 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /root 2024-04-24T22:37:27.547 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2024-04-24T22:37:27.547 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:27.720 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:37:27.736 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:37:27.736 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:37:27.736 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T22:37:27.736 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:28.829 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-24T22:37:28.829 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-24T22:37:29.215 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:37:29.241 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:37:29.241 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:37:29.241 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T22:37:29.241 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:37:29.241 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:37:29.241 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:29.466 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:37:29.489 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:37:29.489 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:37:29.489 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T22:37:29.490 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:37:29.490 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:37:29.490 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:29.933 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:37:29.956 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:37:29.957 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:37:29.957 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T22:37:29.957 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:37:29.957 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:37:29.957 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:30.055 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T22:37:30.095 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T22:37:30.150 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T22:37:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T22:37:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T22:37:30.461 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T22:37:30.551 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:37:30.690 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:37:30.725 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T22:37:30.782 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T22:37:30.810 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T22:37:31.260 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:37:31.292 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:37:31.323 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T22:37:31.394 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T22:37:31.423 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:37:32.450 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:37:32.492 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:37:33.459 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:37:33.484 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:37:33.587 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:37:33.747 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T22:37:33.778 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:37:33.801 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:37:33.801 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:37:33.801 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T22:37:33.802 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:37:33.802 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:37:33.802 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:33.845 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:37:33.856 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:37:33.954 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T22:37:34.001 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T22:37:34.904 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T22:37:34.931 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:37:34.954 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:37:34.954 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:37:34.954 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T22:37:34.954 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:37:34.954 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:37:34.954 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:34.998 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:37:35.020 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:37:35.020 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:37:35.020 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T22:37:35.020 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:35.074 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:37:35.096 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:37:35.096 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:37:35.097 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T22:37:35.097 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:37:35.097 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:37:35.097 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:35.351 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-24T22:37:35.408 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:37:35.431 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:37:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:37:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T22:37:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:37:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:37:35.431 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:35.462 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T22:37:35.521 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T22:37:35.713 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T22:37:35.809 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T22:37:35.857 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T22:37:35.985 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T22:37:36.447 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T22:37:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:37:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:37:36.795 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:37:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:37:37.012 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:37:37.349 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T22:37:37.409 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T22:37:37.433 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:37:37.645 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:37:37.676 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:37:37.833 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T22:37:37.860 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:37:37.925 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T22:37:38.267 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:37:38.320 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T22:37:38.524 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T22:37:38.681 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T22:37:38.725 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T22:37:38.780 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T22:37:38.886 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T22:37:38.965 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T22:37:39.044 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T22:37:39.097 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T22:37:39.136 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T22:37:39.259 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T22:37:39.336 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T22:37:39.378 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T22:37:39.441 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T22:37:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T22:37:39.607 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T22:37:39.666 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T22:37:39.706 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T22:37:39.861 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T22:37:39.913 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T22:37:39.929 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T22:37:39.955 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T22:37:39.995 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T22:37:40.028 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T22:37:40.041 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T22:37:40.055 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T22:37:40.238 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T22:37:40.285 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T22:37:40.315 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T22:37:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T22:37:40.452 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T22:37:40.898 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:37:40.947 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:37:40.967 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:37:41.189 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T22:37:41.295 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T22:37:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T22:37:41.526 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T22:37:41.578 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T22:37:41.655 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T22:37:41.794 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T22:37:41.894 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T22:37:41.923 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T22:37:41.934 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T22:37:42.007 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T22:37:42.015 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T22:37:42.048 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T22:37:42.048 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:37:42.091 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T22:37:42.133 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:37:42.174 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:37:42.174 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:37:42.191 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T22:37:42.262 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T22:37:42.290 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:37:42.314 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T22:37:42.355 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:37:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T22:37:42.571 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T22:37:42.641 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T22:37:42.713 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T22:37:42.922 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T22:37:42.975 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T22:37:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T22:37:43.086 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T22:37:43.231 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T22:37:43.328 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T22:37:43.385 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T22:37:43.453 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T22:37:43.486 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T22:37:43.690 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T22:37:43.878 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T22:37:43.916 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T22:37:43.955 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T22:37:44.011 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T22:37:44.080 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T22:37:44.140 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T22:37:44.187 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T22:37:44.258 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T22:37:44.309 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T22:37:44.387 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T22:37:44.443 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T22:37:44.589 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T22:37:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T22:37:44.696 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T22:37:44.767 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T22:37:44.916 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T22:37:44.975 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T22:37:45.992 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T22:37:46.050 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T22:37:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T22:37:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T22:37:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T22:37:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T22:37:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T22:37:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T22:37:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T22:37:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T22:37:47.735 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T22:37:47.892 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T22:37:48.564 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:37:48.564 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T22:37:48.564 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T22:37:48.564 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T22:37:48.564 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T22:37:48.564 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T22:37:48.564 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T22:37:48.565 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T22:37:48.566 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T22:37:48.567 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T22:37:48.568 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T22:37:48.569 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T22:37:48.570 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T22:37:48.571 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T22:37:48.572 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T22:37:48.573 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T22:37:48.574 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T22:37:48.575 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T22:37:48.576 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T22:37:48.577 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:37:48.577 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T22:37:49.563 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T22:37:49.662 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T22:37:49.722 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T22:37:49.793 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T22:37:50.237 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T22:37:50.548 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:37:50.548 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:50.548 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2024-04-24T22:37:50.548 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.548 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.548 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T22:37:50.548 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T22:37:50.548 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-24T22:37:50.548 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T22:37:50.548 INFO:teuthology.orchestra.run.smithi113.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:37:50.549 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T22:37:50.550 INFO:teuthology.orchestra.run.smithi113.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.551 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T22:37:50.552 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T22:37:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T22:37:50.554 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T22:37:50.555 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T22:37:50.556 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T22:37:50.557 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:37:50.558 INFO:teuthology.orchestra.run.smithi113.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T22:37:50.559 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T22:37:50.559 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T22:37:50.562 INFO:teuthology.orchestra.run.smithi113.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T22:37:50.562 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T22:37:50.562 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T22:37:50.562 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T22:37:50.563 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:37:50.563 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-24T22:37:50.580 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:37:50.744 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T22:37:50.783 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T22:37:50.848 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T22:37:50.965 DEBUG:teuthology.parallel:result is None 2024-04-24T22:37:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T22:37:51.469 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:37:51.562 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T22:37:51.587 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T22:37:54.350 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:37:54.402 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:37:55.090 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T22:37:55.138 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:37:55.292 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T22:37:55.292 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T22:37:55.292 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T22:37:55.293 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:37:55.339 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:38:08.711 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T22:38:08.711 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-04-24T22:38:08.711 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-04-24T22:38:08.711 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-04-24T22:38:08.711 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-04-24T22:38:08.711 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-04-24T22:38:08.711 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-04-24T22:38:08.712 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-04-24T22:38:08.712 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:08.951 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:38:08.997 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T22:38:08.997 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:38:08.998 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T22:38:08.998 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:10.703 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:38:10.812 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T22:38:10.812 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:38:10.813 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T22:38:10.813 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:38:10.813 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T22:38:10.813 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:11.095 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:38:11.192 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T22:38:11.192 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:38:11.193 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T22:38:11.193 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:38:11.193 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T22:38:11.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:11.704 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:38:11.798 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T22:38:11.799 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:38:11.799 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T22:38:11.799 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:38:11.799 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T22:38:11.799 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:11.930 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T22:38:11.992 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T22:38:12.092 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T22:38:12.353 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T22:38:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T22:38:12.532 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T22:38:12.640 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:38:12.811 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:38:12.867 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T22:38:12.917 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T22:38:12.957 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T22:38:13.607 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:38:13.644 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T22:38:13.702 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T22:38:13.818 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T22:38:13.853 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:38:15.209 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T22:38:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:38:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T22:38:16.455 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:38:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T22:38:16.803 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T22:38:16.832 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:38:16.904 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T22:38:16.905 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:38:16.905 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T22:38:16.905 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:38:16.905 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T22:38:16.905 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:16.975 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:38:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T22:38:17.154 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T22:38:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T22:38:18.283 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T22:38:18.344 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:38:18.416 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T22:38:18.417 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:38:18.417 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T22:38:18.417 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:38:18.417 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T22:38:18.417 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:18.469 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:38:18.542 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T22:38:18.542 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:38:18.543 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T22:38:18.543 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:18.620 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:38:18.704 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T22:38:18.704 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:38:18.704 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T22:38:18.704 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:38:18.704 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T22:38:18.704 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:19.120 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:38:19.211 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T22:38:19.211 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T22:38:19.212 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T22:38:19.212 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:38:19.212 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T22:38:19.212 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:24.402 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T22:38:24.453 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T22:38:24.546 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T22:38:24.613 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T22:38:24.664 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T22:38:25.014 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T22:38:25.065 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T22:38:25.093 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T22:38:26.009 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:38:26.057 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:38:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T22:38:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T22:38:27.688 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T22:38:27.791 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T22:38:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T22:38:28.046 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T22:38:28.046 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:38:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T22:38:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:38:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:38:28.383 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T22:38:28.520 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:38:28.817 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:38:28.817 INFO:teuthology.orchestra.run.smithi047.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T22:38:28.817 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:34.395 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T22:38:34.395 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T22:38:34.395 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T22:38:34.396 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T22:38:34.397 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T22:38:34.397 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T22:38:34.397 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T22:38:34.397 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T22:38:34.397 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T22:38:34.397 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T22:38:34.397 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T22:38:34.397 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T22:38:34.410 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T22:38:34.410 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T22:38:34.410 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T22:38:34.410 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T22:38:34.410 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T22:38:34.410 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T22:38:34.410 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T22:38:34.410 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T22:38:34.410 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T22:38:34.411 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T22:38:34.412 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T22:38:34.413 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T22:38:34.414 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T22:38:34.415 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T22:38:34.416 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T22:38:34.417 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T22:38:34.418 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T22:38:34.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T22:38:34.420 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T22:38:34.420 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T22:38:34.420 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T22:38:34.420 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T22:38:34.420 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T22:38:34.420 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T22:38:36.879 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:38:36.880 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T22:38:36.881 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T22:38:36.882 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T22:38:36.883 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T22:38:36.884 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T22:38:36.885 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T22:38:36.886 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T22:38:36.887 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T22:38:36.888 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T22:38:36.889 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T22:38:36.890 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T22:38:36.890 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T22:38:36.890 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T22:38:36.890 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T22:38:36.890 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T22:38:36.890 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T22:38:36.890 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:36.890 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-24T22:38:37.410 DEBUG:teuthology.parallel:result is None 2024-04-24T22:38:37.410 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:38:37.636 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T22:38:37.693 INFO:teuthology.orchestra.run.smithi047.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T22:38:37.694 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T22:38:37.694 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T22:38:37.696 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:38:37.916 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T22:38:37.942 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T22:38:37.942 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T22:38:37.942 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T22:38:37.944 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T22:38:37.944 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:37.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T22:38:38.021 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:38:38.021 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T22:38:38.051 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T22:38:38.051 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:38.051 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T22:38:38.132 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T22:38:38.240 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:38:38.240 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T22:38:38.264 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T22:38:38.328 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T22:38:38.328 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:38.328 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T22:38:38.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T22:38:38.503 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:38:38.503 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T22:38:38.526 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T22:38:38.591 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T22:38:38.591 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:38.591 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T22:38:38.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T22:38:38.768 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:38:38.769 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T22:38:38.792 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T22:38:38.857 INFO:teuthology.run_tasks:Running task ceph... 2024-04-24T22:38:38.967 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-24T22:38:38.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2024-04-24T22:38:38.973 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /var/log/ceph 2024-04-24T22:38:39.038 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-24T22:38:39.038 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T22:38:39.042 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T22:38:39.104 INFO:tasks.ceph:Creating extra log directories... 2024-04-24T22:38:39.104 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T22:38:39.109 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T22:38:39.179 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-24T22:38:39.179 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-24T22:38:39.180 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-24T22:38:39.180 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:39.180 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T22:38:39.222 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T22:38:39.410 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T22:38:39.475 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:38:39.475 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T22:38:39.501 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T22:38:39.624 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T22:38:39.667 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-24T22:38:39.667 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-24T22:38:39.667 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135', 'branch': 'squid', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672135', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi047.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFOUYB/JG2fAyFqtbBUdOcyVMq3GimtEw9l8u24AALYq0jtgM3DKmD8AYRbbZ58s3yUDHqiyZktcO+CGoMuyWNE=', 'smithi113.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNuj/FFs27X8PNYyAYlZ7ag45pEvHd0Kr6FrNHG/Nr4cnOjY0h9VgFE20HO2RsC2/N7/A9mQ4R81JSR417+VSzY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['suites/iozone.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-24T22:38:39.668 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T22:38:39.679 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T22:38:39.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T22:38:39.767 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T22:38:39.835 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:39.835 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T22:38:39.878 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T22:38:39.879 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-04-24T22:38:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T22:38:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:39.967 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-24T22:38:39.967 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:39.967 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:39.967 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-24 22:38:29.348785596 +0000 2024-04-24T22:38:39.967 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-24 22:35:49.729794049 +0000 2024-04-24T22:38:39.967 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-24 22:35:49.729794049 +0000 2024-04-24T22:38:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-24 22:35:49.729794049 +0000 2024-04-24T22:38:39.968 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T22:38:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-24T22:38:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-24T22:38:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000315051 s, 1.6 MB/s 2024-04-24T22:38:40.078 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T22:38:40.120 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-04-24T22:38:40.202 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T22:38:40.202 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:40.202 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-24T22:38:40.202 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:40.202 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:40.202 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-24 22:38:29.349785576 +0000 2024-04-24T22:38:40.203 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-24 22:35:50.320781673 +0000 2024-04-24T22:38:40.203 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-24 22:35:50.320781673 +0000 2024-04-24T22:38:40.203 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-24 22:35:50.320781673 +0000 2024-04-24T22:38:40.203 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T22:38:40.302 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-24T22:38:40.302 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-24T22:38:40.302 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000193431 s, 2.6 MB/s 2024-04-24T22:38:40.305 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T22:38:40.348 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-04-24T22:38:40.429 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T22:38:40.429 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:40.430 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-24T22:38:40.430 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:40.430 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:40.430 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-24 22:38:29.349785576 +0000 2024-04-24T22:38:40.430 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-24 22:35:50.958768313 +0000 2024-04-24T22:38:40.430 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-24 22:35:50.958768313 +0000 2024-04-24T22:38:40.430 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-24 22:35:50.958768313 +0000 2024-04-24T22:38:40.430 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T22:38:40.531 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-24T22:38:40.531 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-24T22:38:40.531 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000211363 s, 2.4 MB/s 2024-04-24T22:38:40.534 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T22:38:40.575 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-04-24T22:38:40.657 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T22:38:40.657 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:40.657 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-24T22:38:40.657 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:40.657 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:40.657 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-24 22:38:29.350785554 +0000 2024-04-24T22:38:40.657 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-24 22:35:51.583755225 +0000 2024-04-24T22:38:40.657 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-24 22:35:51.583755225 +0000 2024-04-24T22:38:40.657 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-24 22:35:51.583755225 +0000 2024-04-24T22:38:40.658 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T22:38:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-24T22:38:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-24T22:38:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000193721 s, 2.6 MB/s 2024-04-24T22:38:40.765 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T22:38:40.807 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T22:38:40.807 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:38:40.807 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T22:38:40.826 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T22:38:40.826 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-04-24T22:38:40.883 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T22:38:40.883 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:40.883 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-24T22:38:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-24 22:37:42.713010554 +0000 2024-04-24T22:38:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-24 22:34:58.543702790 +0000 2024-04-24T22:38:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-24 22:34:58.543702790 +0000 2024-04-24T22:38:40.884 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-24 22:34:58.543702790 +0000 2024-04-24T22:38:40.884 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T22:38:40.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-24T22:38:40.949 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-24T22:38:40.949 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000201452 s, 2.5 MB/s 2024-04-24T22:38:40.951 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T22:38:41.008 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-04-24T22:38:41.064 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T22:38:41.064 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:41.064 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-24T22:38:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-24 22:37:42.713010554 +0000 2024-04-24T22:38:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-24 22:34:58.892695136 +0000 2024-04-24T22:38:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-24 22:34:58.892695136 +0000 2024-04-24T22:38:41.065 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-24 22:34:58.892695136 +0000 2024-04-24T22:38:41.066 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T22:38:41.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-24T22:38:41.128 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-24T22:38:41.128 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000223812 s, 2.3 MB/s 2024-04-24T22:38:41.129 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T22:38:41.185 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-04-24T22:38:41.241 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T22:38:41.241 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-24T22:38:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-24 22:37:42.714010532 +0000 2024-04-24T22:38:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-24 22:34:59.267686912 +0000 2024-04-24T22:38:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-24 22:34:59.267686912 +0000 2024-04-24T22:38:41.241 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-24 22:34:59.267686912 +0000 2024-04-24T22:38:41.241 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T22:38:41.303 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-24T22:38:41.304 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-24T22:38:41.304 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000247885 s, 2.1 MB/s 2024-04-24T22:38:41.305 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T22:38:41.361 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-04-24T22:38:41.417 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T22:38:41.417 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T22:38:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-24T22:38:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:38:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T22:38:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-24 22:37:42.714010532 +0000 2024-04-24T22:38:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-24 22:34:59.617679237 +0000 2024-04-24T22:38:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-24 22:34:59.617679237 +0000 2024-04-24T22:38:41.417 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-24 22:34:59.617679237 +0000 2024-04-24T22:38:41.418 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T22:38:41.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-24T22:38:41.479 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-24T22:38:41.479 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000217414 s, 2.4 MB/s 2024-04-24T22:38:41.480 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T22:38:41.536 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T22:38:41.537 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T22:38:41.537 INFO:tasks.ceph:Generating config... 2024-04-24T22:38:41.538 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-24T22:38:41.539 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-24T22:38:41.539 INFO:tasks.ceph:[client] debug client = 20 2024-04-24T22:38:41.539 INFO:tasks.ceph:[client] debug ms = 1 2024-04-24T22:38:41.539 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-24T22:38:41.539 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-24T22:38:41.539 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-24T22:38:41.539 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-24T22:38:41.539 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-24T22:38:41.539 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-24T22:38:41.539 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-24T22:38:41.540 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-24T22:38:41.540 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-24T22:38:41.540 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-24T22:38:41.540 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-24T22:38:41.540 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-24T22:38:41.540 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-04-24T22:38:41.540 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-24T22:38:41.540 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-24T22:38:41.540 INFO:tasks.ceph:[mds] mds_dir_prefetch = False 2024-04-24T22:38:41.541 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-04-24T22:38:41.541 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-24T22:38:41.541 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-24T22:38:41.541 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-24T22:38:41.541 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-24T22:38:41.541 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-24T22:38:41.541 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-24T22:38:41.541 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-24T22:38:41.542 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-24T22:38:41.542 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-24T22:38:41.542 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-24T22:38:41.542 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-24T22:38:41.542 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-24T22:38:41.542 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-24T22:38:41.542 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-24T22:38:41.542 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-24T22:38:41.542 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-24T22:38:41.542 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-24T22:38:41.542 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-24T22:38:41.542 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-24T22:38:41.543 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-24T22:38:41.543 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-24T22:38:41.543 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-24T22:38:41.543 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-24T22:38:41.543 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-24T22:38:41.543 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-24T22:38:41.543 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-24T22:38:41.543 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-24T22:38:41.543 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-24T22:38:41.543 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-24T22:38:41.543 INFO:tasks.ceph:Setting up mon.a... 2024-04-24T22:38:41.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-24T22:38:41.655 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.keyring 2024-04-24T22:38:41.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-24T22:38:41.754 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T22:38:41.817 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.47'), ('mon.b', '172.21.15.113'), ('mon.c', '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]')] 2024-04-24T22:38:41.817 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.47,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-24T22:38:41.818 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:41.819 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T22:38:41.856 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.47 --add b 172.21.15.113 --addv c '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:38:41.977 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:38:41.977 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: generated fsid 617410c3-1ae2-44fe-8492-d21ee235c3cc 2024-04-24T22:38:41.977 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = quincy 2024-04-24T22:38:41.977 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 0 2024-04-24T22:38:41.977 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 617410c3-1ae2-44fe-8492-d21ee235c3cc 2024-04-24T22:38:41.977 INFO:teuthology.orchestra.run.smithi047.stdout:last_changed 2024-04-24T22:38:41.976658+0000 2024-04-24T22:38:41.978 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-04-24T22:38:41.976658+0000 2024-04-24T22:38:41.978 INFO:teuthology.orchestra.run.smithi047.stdout:min_mon_release 17 (quincy) 2024-04-24T22:38:41.978 INFO:teuthology.orchestra.run.smithi047.stdout:election_strategy: 1 2024-04-24T22:38:41.978 INFO:teuthology.orchestra.run.smithi047.stdout:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-24T22:38:41.978 INFO:teuthology.orchestra.run.smithi047.stdout:1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-04-24T22:38:41.978 INFO:teuthology.orchestra.run.smithi047.stdout:2: [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] mon.c 2024-04-24T22:38:41.978 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-24T22:38:41.980 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T22:38:42.018 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 617410c3-1ae2-44fe-8492-d21ee235c3cc... 2024-04-24T22:38:42.019 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T22:38:42.062 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T22:38:42.106 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-04-24T22:38:42.106 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-04-24T22:38:42.107 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-04-24T22:38:42.108 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.109 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-04-24T22:38:42.110 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-04-24T22:38:42.110 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-04-24T22:38:42.110 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.47,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2024-04-24T22:38:42.110 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on skipped message = False 2024-04-24T22:38:42.110 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = 617410c3-1ae2-44fe-8492-d21ee235c3cc 2024-04-24T22:38:42.110 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.110 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-04-24T22:38:42.110 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-04-24T22:38:42.110 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-04-24T22:38:42.111 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.112 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-04-24T22:38:42.112 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.112 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-04-24T22:38:42.112 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.112 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-04-24T22:38:42.112 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.112 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-04-24T22:38:42.112 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-04-24T22:38:42.112 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: bdev async discard = True 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: bdev enable discard = True 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore allocator = bitmap 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 1/20 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 1/20 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 4/10 2024-04-24T22:38:42.113 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T22:38:42.114 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-04-24T22:38:42.114 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-04-24T22:38:42.114 INFO:teuthology.orchestra.run.smithi113.stdout: osd blocked scrub grace period = 3600 2024-04-24T22:38:42.114 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2024-04-24T22:38:42.114 INFO:teuthology.orchestra.run.smithi113.stdout: osd max markdown count = 1000 2024-04-24T22:38:42.114 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2024-04-24T22:38:42.114 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-24T22:38:42.114 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-04-24T22:38:42.115 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-24T22:38:42.116 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-04-24T22:38:42.116 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-04-24T22:38:42.116 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-24T22:38:42.116 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-04-24T22:38:42.116 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-04-24T22:38:42.116 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-24T22:38:42.116 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.116 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: mon down mkfs grace = 300 2024-04-24T22:38:42.117 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2024-04-24T22:38:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-24T22:38:42.118 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-04-24T22:38:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-04-24T22:38:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-04-24T22:38:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-04-24T22:38:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T22:38:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T22:38:42.118 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2024-04-24T22:38:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: client_shutdown_timeout = 120 2024-04-24T22:38:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-04-24T22:38:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-24T22:38:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-24T22:38:42.119 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-24T22:38:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-04-24T22:38:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-04-24T22:38:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-04-24T22:38:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-04-24T22:38:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-04-24T22:38:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.e] 2024-04-24T22:38:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-04-24T22:38:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-04-24T22:38:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.f] 2024-04-24T22:38:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2024-04-24T22:38:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2024-04-24T22:38:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2024-04-24T22:38:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-24T22:38:42.120 INFO:teuthology.orchestra.run.smithi113.stdout: mds bal fragment size max = 10000 2024-04-24T22:38:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: mds bal merge size = 5 2024-04-24T22:38:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: mds bal split bits = 3 2024-04-24T22:38:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: mds bal split size = 100 2024-04-24T22:38:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2024-04-24T22:38:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = True 2024-04-24T22:38:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: mds oft prefetch dirfrags = True 2024-04-24T22:38:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2024-04-24T22:38:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = True 2024-04-24T22:38:42.121 INFO:teuthology.orchestra.run.smithi113.stdout: mds_dir_prefetch = False 2024-04-24T22:38:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: mds_sleep_rank_change = 5000000.0 2024-04-24T22:38:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-04-24T22:38:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-04-24T22:38:42.122 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-04-24T22:38:42.165 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-04-24T22:38:42.165 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-04-24T22:38:42.165 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T22:38:42.165 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-04-24T22:38:42.165 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.165 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T22:38:42.166 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-04-24T22:38:42.167 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on skipped message = False 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 617410c3-1ae2-44fe-8492-d21ee235c3cc 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.168 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug trim objects = true 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-04-24T22:38:42.169 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-04-24T22:38:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: bdev async discard = True 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: bdev enable discard = True 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = bitmap 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 1/20 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 1/20 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 4/10 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: osd blocked scrub grace period = 3600 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-04-24T22:38:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: osd max markdown count = 1000 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-04-24T22:38:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T22:38:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: mon down mkfs grace = 300 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-04-24T22:38:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: client_shutdown_timeout = 120 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-04-24T22:38:42.175 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.e] 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.f] 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds balancer = 20 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal fragment size max = 10000 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal merge size = 5 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal split bits = 3 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: mds bal split size = 100 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2024-04-24T22:38:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2024-04-24T22:38:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: mds oft prefetch dirfrags = True 2024-04-24T22:38:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2024-04-24T22:38:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2024-04-24T22:38:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: mds_dir_prefetch = False 2024-04-24T22:38:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: mds_sleep_rank_change = 5000000.0 2024-04-24T22:38:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-04-24T22:38:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-04-24T22:38:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-04-24T22:38:42.191 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-24T22:38:42.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-24T22:38:42.300 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-24T22:38:42.300 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:42.300 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-24T22:38:42.339 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:42.340 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-24T22:38:42.433 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:38:42.434 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:42.434 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T22:38:42.434 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T22:38:42.534 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:38:42.534 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:38:42.618 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:38:42.618 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:38:42.618 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T22:38:42.618 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T22:38:42.656 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:38:42.657 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:38:42.713 INFO:tasks.ceph:Setting up mon nodes... 2024-04-24T22:38:42.713 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-24T22:38:42.713 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T22:38:42.861 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T22:38:42.869 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T22:38:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T22:38:42.935 INFO:tasks.ceph:Setting up mds nodes... 2024-04-24T22:38:42.935 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T22:38:43.056 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T22:38:43.061 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-24T22:38:43.126 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T22:38:43.236 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T22:38:43.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-24T22:38:43.304 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-04-24T22:38:43.419 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-04-24T22:38:43.425 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-04-24T22:38:43.486 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T22:38:43.541 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T22:38:43.544 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-24T22:38:43.569 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T22:38:43.660 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T22:38:43.662 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-24T22:38:43.686 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-04-24T22:38:43.775 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-04-24T22:38:43.777 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-04-24T22:38:43.800 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-24T22:38:43.800 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T22:38:43.901 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-24T22:38:43.939 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-24T22:38:43.939 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T22:38:43.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-24T22:38:44.006 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T22:38:44.006 INFO:tasks.ceph:role: osd.0 2024-04-24T22:38:44.006 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:38:44.006 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T22:38:44.069 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:38:44.069 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:38:44.069 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:38:44.069 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:38:44.069 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:38:44.069 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:38:44.069 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:38:44.070 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:38:44.070 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:38:44.070 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:38:44.102 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-24T22:38:44.107 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-24T22:38:44.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-24T22:38:44.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-24T22:38:44.265 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-24T22:38:44.327 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T22:38:44.327 INFO:tasks.ceph:role: osd.1 2024-04-24T22:38:44.327 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:38:44.327 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T22:38:44.384 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:38:44.385 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:38:44.385 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:38:44.385 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:38:44.385 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:38:44.385 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:38:44.385 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:38:44.385 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:38:44.385 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:38:44.385 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:38:44.419 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-24T22:38:44.423 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-24T22:38:44.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-24T22:38:44.514 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-24T22:38:44.583 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-24T22:38:44.645 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T22:38:44.645 INFO:tasks.ceph:role: osd.2 2024-04-24T22:38:44.645 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:38:44.645 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T22:38:44.703 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:38:44.703 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:38:44.703 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:38:44.703 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:38:44.704 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:38:44.704 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:38:44.704 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:38:44.704 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:38:44.704 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:38:44.704 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:38:44.738 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-24T22:38:44.743 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-24T22:38:44.743 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-24T22:38:44.832 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-24T22:38:44.896 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-24T22:38:44.956 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T22:38:44.956 INFO:tasks.ceph:role: osd.3 2024-04-24T22:38:44.956 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:38:44.956 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T22:38:45.017 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:38:45.018 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:38:45.018 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:38:45.018 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:38:45.018 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:38:45.018 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:38:45.018 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:38:45.018 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:38:45.018 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:38:45.018 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:38:45.051 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-24T22:38:45.056 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-24T22:38:45.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-24T22:38:45.154 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-24T22:38:45.220 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:38:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:45.415+0000 7f6481eee640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-24T22:38:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:45.416+0000 7f6481eee640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-24T22:38:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:45.416+0000 7f6481eee640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T22:38:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:45.416+0000 7f6481eee640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T22:38:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:45.416+0000 7f6481eee640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T22:38:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:45.418+0000 7f6481eee640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-24T22:38:48.644 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T22:38:48.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:38:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:48.876+0000 7feae5a18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-24T22:38:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:48.876+0000 7feae5a18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-24T22:38:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:48.876+0000 7feae5a18640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T22:38:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:48.877+0000 7feae5a18640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T22:38:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:48.877+0000 7feae5a18640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T22:38:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:48.879+0000 7feae5a18640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-24T22:38:52.117 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T22:38:52.218 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:38:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:52.356+0000 7fc06ca0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-24T22:38:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:52.357+0000 7fc06ca0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-24T22:38:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:52.357+0000 7fc06ca0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T22:38:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:52.357+0000 7fc06ca0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T22:38:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:52.357+0000 7fc06ca0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T22:38:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:52.360+0000 7fc06ca0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-24T22:38:55.576 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T22:38:55.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:38:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:55.808+0000 7fcbbefdc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-24T22:38:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:55.809+0000 7fcbbefdc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-24T22:38:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:55.809+0000 7fcbbefdc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T22:38:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:55.809+0000 7fcbbefdc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T22:38:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:55.809+0000 7fcbbefdc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T22:38:55.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:38:55.811+0000 7fcbbefdc640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-24T22:38:58.789 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T22:38:58.878 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-24T22:38:58.907 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T22:38:58.907 INFO:tasks.ceph:role: osd.4 2024-04-24T22:38:58.907 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:38:58.907 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T22:38:58.982 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:38:58.982 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:38:58.982 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:38:58.982 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:38:58.982 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:38:58.982 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:38:58.982 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:38:58.982 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:38:58.982 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:38:58.982 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:38:59.497 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-24T22:38:59.500 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-24T22:38:59.500 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-24T22:38:59.592 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-24T22:38:59.623 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-24T22:38:59.689 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T22:38:59.689 INFO:tasks.ceph:role: osd.5 2024-04-24T22:38:59.689 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:38:59.689 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T22:38:59.763 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:38:59.763 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:38:59.763 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:38:59.763 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:38:59.764 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:38:59.764 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:38:59.764 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:38:59.764 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:38:59.764 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:38:59.764 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:39:00.187 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-24T22:39:00.189 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-24T22:39:00.190 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-24T22:39:00.258 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-24T22:39:00.326 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-24T22:39:00.389 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T22:39:00.389 INFO:tasks.ceph:role: osd.6 2024-04-24T22:39:00.389 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:39:00.390 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T22:39:00.460 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:39:00.460 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:39:00.460 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:39:00.460 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:39:00.460 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:39:00.460 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:39:00.460 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:39:00.460 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:39:00.461 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:39:00.461 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:39:00.966 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-24T22:39:00.969 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-24T22:39:00.969 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-24T22:39:01.061 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-24T22:39:01.091 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-24T22:39:01.155 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T22:39:01.155 INFO:tasks.ceph:role: osd.7 2024-04-24T22:39:01.155 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:39:01.155 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T22:39:01.227 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T22:39:01.227 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T22:39:01.227 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T22:39:01.227 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T22:39:01.227 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T22:39:01.227 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-24T22:39:01.227 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T22:39:01.227 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T22:39:01.228 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T22:39:01.228 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T22:39:01.749 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-24T22:39:01.750 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-24T22:39:01.751 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-24T22:39:01.857 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-24T22:39:01.886 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:39:01.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:01.974+0000 7f0e2e243640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-24T22:39:01.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:01.976+0000 7f0e2e243640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-24T22:39:01.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:01.976+0000 7f0e2e243640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T22:39:01.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:01.976+0000 7f0e2e243640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T22:39:01.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:01.976+0000 7f0e2e243640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T22:39:01.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:01.980+0000 7f0e2e243640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-24T22:39:04.840 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T22:39:04.866 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:39:04.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:04.952+0000 7fa8881cb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-24T22:39:04.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:04.952+0000 7fa8881cb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-24T22:39:04.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:04.952+0000 7fa8881cb640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T22:39:04.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:04.952+0000 7fa8881cb640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T22:39:04.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:04.952+0000 7fa8881cb640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T22:39:04.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:04.953+0000 7fa8881cb640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-24T22:39:07.809 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T22:39:07.838 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:39:07.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:07.925+0000 7fdfd61d6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-24T22:39:07.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:07.925+0000 7fdfd61d6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-24T22:39:07.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:07.925+0000 7fdfd61d6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T22:39:07.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:07.925+0000 7fdfd61d6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T22:39:07.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:07.925+0000 7fdfd61d6640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T22:39:07.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:07.926+0000 7fdfd61d6640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-24T22:39:09.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:39:09.677 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:39:10.775 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-24T22:39:10.800 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:39:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:10.885+0000 7f66f2d4c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-24T22:39:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:10.885+0000 7f66f2d4c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-24T22:39:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:10.885+0000 7f66f2d4c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T22:39:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:10.885+0000 7f66f2d4c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T22:39:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:10.885+0000 7f66f2d4c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T22:39:10.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:10.887+0000 7f66f2d4c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-24T22:39:13.745 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-24T22:39:13.772 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-24T22:39:13.773 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:13.773 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-24T22:39:13.868 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:13.868 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-24T22:39:13.957 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:13.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-24T22:39:14.046 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:14.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-04-24T22:39:14.135 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:14.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-24T22:39:14.225 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:14.225 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-24T22:39:14.309 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:14.309 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-24T22:39:14.400 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:14.401 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-24T22:39:14.488 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:14.488 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-24T22:39:14.517 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:14.517 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-24T22:39:14.585 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:14.585 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-24T22:39:14.656 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:14.656 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-04-24T22:39:14.722 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:14.723 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-24T22:39:14.791 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:14.792 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-24T22:39:14.860 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:14.860 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-24T22:39:14.925 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:14.925 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-24T22:39:14.990 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:14.990 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-24T22:39:15.051 INFO:tasks.ceph:Adding keys to all mons... 2024-04-24T22:39:15.052 DEBUG:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T22:39:15.061 DEBUG:teuthology.orchestra.run.smithi113:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T22:39:15.084 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.y] 2024-04-24T22:39:15.084 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDyiSlmIOBOMxAAHs1pwJbu+OEIqVIjenfT4w== 2024-04-24T22:39:15.084 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-04-24T22:39:15.084 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDziSlmT/hQAxAAtAfl0D72qkGHGljjC/UoCw== 2024-04-24T22:39:15.084 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-04-24T22:39:15.084 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDziSlmXbsODhAA6A07yxZJmNg+0A4j3lJziQ== 2024-04-24T22:39:15.084 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.e] 2024-04-24T22:39:15.084 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDziSlm8k38GBAAjEqQnFEdVBpZgR+iN20Sdg== 2024-04-24T22:39:15.085 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.0] 2024-04-24T22:39:15.085 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD1iSlmCYDOGBAAT2LXaTynDT920MDmYa/ApQ== 2024-04-24T22:39:15.085 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.1] 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD4iSlmmY5GNBAA7JEG14WvC1ghfNqTv/Z+QA== 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.2] 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD8iSlm3MRLFRAALCatYNiR8KwmpZTupzbOBg== 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.3] 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD/iSlm2iw5MBAAKfZQqqhoyinCr7KmqcbQag== 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.x] 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDyiSlmjsqLNxAAr85C9cmedN9EefCqe1ZpWA== 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDziSlmKBI/IBAAsaQmVV2Sxvg1/KloqNlaww== 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDziSlm90VNJxAAL6Dv9ceYiE8LESxjDGhbuA== 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.f] 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDziSlmHLMtLhAAMnr7yClU6lEyxPfehKBoKQ== 2024-04-24T22:39:15.086 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.4] 2024-04-24T22:39:15.087 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAFiilmD3sgOhAAK7WHvP3ZGPbkMWM9/xFW7w== 2024-04-24T22:39:15.087 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.5] 2024-04-24T22:39:15.087 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAIiilmh9LXOBAAsPgNrXJCZBCF4whhsoIm1Q== 2024-04-24T22:39:15.087 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.6] 2024-04-24T22:39:15.087 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQALiilmg/Q4NxAAJrGPO2Wcm9TVmYApbjRhsw== 2024-04-24T22:39:15.087 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.7] 2024-04-24T22:39:15.087 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAOiilmeY3UNBAA7RMH63/kzkbfeku2xGRZyQ== 2024-04-24T22:39:15.087 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-24T22:39:15.087 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQDziSlmtreyNRAAO0Z2H71YmeRp73bFHSDyww== 2024-04-24T22:39:15.142 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.y] 2024-04-24T22:39:15.142 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDyiSlmIOBOMxAAHs1pwJbu+OEIqVIjenfT4w== 2024-04-24T22:39:15.142 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDziSlmT/hQAxAAtAfl0D72qkGHGljjC/UoCw== 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDziSlmXbsODhAA6A07yxZJmNg+0A4j3lJziQ== 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.e] 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDziSlm8k38GBAAjEqQnFEdVBpZgR+iN20Sdg== 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQD1iSlmCYDOGBAAT2LXaTynDT920MDmYa/ApQ== 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQD4iSlmmY5GNBAA7JEG14WvC1ghfNqTv/Z+QA== 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQD8iSlm3MRLFRAALCatYNiR8KwmpZTupzbOBg== 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQD/iSlm2iw5MBAAKfZQqqhoyinCr7KmqcbQag== 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2024-04-24T22:39:15.143 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDyiSlmjsqLNxAAr85C9cmedN9EefCqe1ZpWA== 2024-04-24T22:39:15.144 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-04-24T22:39:15.144 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDziSlmKBI/IBAAsaQmVV2Sxvg1/KloqNlaww== 2024-04-24T22:39:15.144 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-04-24T22:39:15.144 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDziSlm90VNJxAAL6Dv9ceYiE8LESxjDGhbuA== 2024-04-24T22:39:15.144 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.f] 2024-04-24T22:39:15.144 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDziSlmHLMtLhAAMnr7yClU6lEyxPfehKBoKQ== 2024-04-24T22:39:15.144 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.4] 2024-04-24T22:39:15.144 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAFiilmD3sgOhAAK7WHvP3ZGPbkMWM9/xFW7w== 2024-04-24T22:39:15.145 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.5] 2024-04-24T22:39:15.145 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAIiilmh9LXOBAAsPgNrXJCZBCF4whhsoIm1Q== 2024-04-24T22:39:15.145 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.6] 2024-04-24T22:39:15.145 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQALiilmg/Q4NxAAJrGPO2Wcm9TVmYApbjRhsw== 2024-04-24T22:39:15.145 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.7] 2024-04-24T22:39:15.145 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAOiilmeY3UNBAA7RMH63/kzkbfeku2xGRZyQ== 2024-04-24T22:39:15.146 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-24T22:39:15.146 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDziSlmtreyNRAAO0Z2H71YmeRp73bFHSDyww== 2024-04-24T22:39:15.147 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T22:39:15.198 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T22:39:15.331 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:15.338 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:15.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:15.476 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:15.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:15.613 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:15.749 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:15.757 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:15.879 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:15.889 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:16.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:16.031 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:16.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:16.170 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:16.310 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T22:39:16.320 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T22:39:16.457 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:16.462 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:16.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:16.610 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:16.745 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:16.758 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T22:39:16.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:16.909 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:17.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:17.057 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:17.191 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:17.198 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:17.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:17.341 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T22:39:17.476 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T22:39:17.484 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T22:39:17.616 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-24T22:39:17.617 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-24T22:39:17.706 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T22:39:17.850 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-24T22:39:17.936 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-24T22:39:17.961 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T22:39:18.056 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-24T22:39:18.079 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-24T22:39:18.143 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T22:39:18.236 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-24T22:39:18.260 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:39:18.268 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T22:39:18.321 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-24T22:39:18.321 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-24T22:39:18.322 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-24T22:39:18.327 INFO:tasks.ceph.mon.a:Started 2024-04-24T22:39:18.327 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-24T22:39:18.327 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-24T22:39:18.358 INFO:tasks.ceph.mon.b:Started 2024-04-24T22:39:18.358 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T22:39:18.359 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-24T22:39:18.361 INFO:tasks.ceph.mon.c:Started 2024-04-24T22:39:18.361 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-24T22:39:18.361 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-24T22:39:18.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-24T22:39:18.366 INFO:tasks.ceph.mgr.y:Started 2024-04-24T22:39:18.366 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-24T22:39:18.366 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-24T22:39:18.368 INFO:tasks.ceph.mgr.x:Started 2024-04-24T22:39:18.368 INFO:tasks.ceph:Setting crush tunables to default 2024-04-24T22:39:18.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd crush tunables default 2024-04-24T22:39:18.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.546+0000 7f3c3ca00640 1 Processor -- start 2024-04-24T22:39:18.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.546+0000 7f3c3ca00640 1 -- start start 2024-04-24T22:39:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.547+0000 7f3c3ca00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3c380cbde0 0x7f3c380ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:18.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.547+0000 7f3c3ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c38073380 0x7f3c380ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.547+0000 7f3c3ca00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c3815b240 con 0x7f3c380cc840 2024-04-24T22:39:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.547+0000 7f3c3ca00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3c3815daa0 con 0x7f3c380cbde0 2024-04-24T22:39:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.547+0000 7f3c3ca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3c3815dcf0 con 0x7f3c38073380 2024-04-24T22:39:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.547+0000 7f3c35400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3c380cbde0 0x7f3c380ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.547+0000 7f3c36800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c38073380 0x7f3c380ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.547+0000 7f3c35400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3c380cbde0 0x7f3c380ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54104/0 (socket says 172.21.15.47:54104) 2024-04-24T22:39:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.547+0000 7f3c35400640 1 -- 172.21.15.47:0/3543287797 learned_addr learned my addr 172.21.15.47:0/3543287797 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.548+0000 7f3c35400640 1 -- 172.21.15.47:0/3543287797 >> v2:172.21.15.113:3300/0 conn(0x7f3c38073380 msgr2=0x7f3c380ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.548+0000 7f3c35400640 1 --2- 172.21.15.47:0/3543287797 >> v2:172.21.15.113:3300/0 conn(0x7f3c38073380 0x7f3c380ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.548+0000 7f3c35400640 1 -- 172.21.15.47:0/3543287797 >> v1:172.21.15.47:6789/0 conn(0x7f3c380cc840 legacy=0x7f3c380cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.548+0000 7f3c35400640 1 -- 172.21.15.47:0/3543287797 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3815dfd0 con 0x7f3c380cbde0 2024-04-24T22:39:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.548+0000 7f3c36800640 1 --2- 172.21.15.47:0/3543287797 >> v2:172.21.15.113:3300/0 conn(0x7f3c38073380 0x7f3c380ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.549+0000 7f3c35400640 1 --2- 172.21.15.47:0/3543287797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c380cbde0 0x7f3c380ca7d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3c3808d640 tx=0x7f3c2402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b420cdfadb3127 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.550+0000 7f3c34a00640 1 -- 172.21.15.47:0/3543287797 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24037070 con 0x7f3c380cbde0 2024-04-24T22:39:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.550+0000 7f3c34a00640 1 -- 172.21.15.47:0/3543287797 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c24032440 con 0x7f3c380cbde0 2024-04-24T22:39:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.550+0000 7f3c34a00640 1 -- 172.21.15.47:0/3543287797 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24032810 con 0x7f3c380cbde0 2024-04-24T22:39:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.550+0000 7f3c3ca00640 1 -- 172.21.15.47:0/3543287797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c380cbde0 msgr2=0x7f3c380ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.550+0000 7f3c3ca00640 1 --2- 172.21.15.47:0/3543287797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c380cbde0 0x7f3c380ca7d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3c3808d640 tx=0x7f3c2402a1f0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.551+0000 7f3c3ca00640 1 -- 172.21.15.47:0/3543287797 shutdown_connections 2024-04-24T22:39:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.551+0000 7f3c3ca00640 1 --2- 172.21.15.47:0/3543287797 >> v2:172.21.15.113:3300/0 conn(0x7f3c38073380 0x7f3c380ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.551+0000 7f3c3ca00640 1 --2- 172.21.15.47:0/3543287797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c380cbde0 0x7f3c380ca7d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.551+0000 7f3c3ca00640 1 -- 172.21.15.47:0/3543287797 >> 172.21.15.47:0/3543287797 conn(0x7f3c3806c940 msgr2=0x7f3c38072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.551+0000 7f3c3ca00640 1 -- 172.21.15.47:0/3543287797 shutdown_connections 2024-04-24T22:39:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.551+0000 7f3c3ca00640 1 -- 172.21.15.47:0/3543287797 wait complete. 2024-04-24T22:39:18.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.552+0000 7f3c3ca00640 1 Processor -- start 2024-04-24T22:39:18.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.553+0000 7f3c3ca00640 1 -- start start 2024-04-24T22:39:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.553+0000 7f3c3ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c38073380 0x7f3c38140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c3ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c380cbde0 0x7f3c381e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c3ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c380cc840 0x7f3c381e5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c3ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3815e870 con 0x7f3c38073380 2024-04-24T22:39:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c3ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3815e590 con 0x7f3c380cbde0 2024-04-24T22:39:18.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c3ca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3c3815af30 con 0x7f3c380cc840 2024-04-24T22:39:18.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c35400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c380cbde0 0x7f3c381e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:18.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c35400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c380cbde0 0x7f3c381e4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:46528/0 (socket says 172.21.15.47:46528) 2024-04-24T22:39:18.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c35400640 1 -- 172.21.15.47:0/64094335 learned_addr learned my addr 172.21.15.47:0/64094335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:18.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c35e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c38073380 0x7f3c38140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:18.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c36800640 1 --2- 172.21.15.47:0/64094335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c380cc840 0x7f3c381e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:18.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c35400640 1 -- 172.21.15.47:0/64094335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c380cc840 msgr2=0x7f3c381e5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:18.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c35400640 1 --2- 172.21.15.47:0/64094335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c380cc840 0x7f3c381e5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c35400640 1 -- 172.21.15.47:0/64094335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c38073380 msgr2=0x7f3c38140b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:18.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c35400640 1 --2- 172.21.15.47:0/64094335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c38073380 0x7f3c38140b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.554+0000 7f3c35400640 1 -- 172.21.15.47:0/64094335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c380ca8d0 con 0x7f3c380cbde0 2024-04-24T22:39:18.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.555+0000 7f3c35400640 1 --2- 172.21.15.47:0/64094335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c380cbde0 0x7f3c381e4c00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3c2402a230 tx=0x7f3c240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:18.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.555+0000 7f3c2e000640 1 -- 172.21.15.47:0/64094335 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24037040 con 0x7f3c380cbde0 2024-04-24T22:39:18.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.555+0000 7f3c2e000640 1 -- 172.21.15.47:0/64094335 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c24042070 con 0x7f3c380cbde0 2024-04-24T22:39:18.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.555+0000 7f3c2e000640 1 -- 172.21.15.47:0/64094335 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240308a0 con 0x7f3c380cbde0 2024-04-24T22:39:18.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.555+0000 7f3c3ca00640 1 -- 172.21.15.47:0/64094335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3815bbd0 con 0x7f3c380cbde0 2024-04-24T22:39:18.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.555+0000 7f3c3ca00640 1 -- 172.21.15.47:0/64094335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3815c190 con 0x7f3c380cbde0 2024-04-24T22:39:18.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.555+0000 7f3c2e000640 1 -- 172.21.15.47:0/64094335 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c24030a40 con 0x7f3c380cbde0 2024-04-24T22:39:18.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.555+0000 7f3c2e000640 1 -- 172.21.15.47:0/64094335 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3c24047af0 con 0x7f3c380cbde0 2024-04-24T22:39:18.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.556+0000 7f3c3ca00640 1 -- 172.21.15.47:0/64094335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c38073b90 con 0x7f3c380cbde0 2024-04-24T22:39:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.559+0000 7f3c2e000640 1 -- 172.21.15.47:0/64094335 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3c24047460 con 0x7f3c380cbde0 2024-04-24T22:39:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.633+0000 7f3c3ca00640 1 -- 172.21.15.47:0/64094335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3c3814b250 con 0x7f3c380cbde0 2024-04-24T22:39:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.638+0000 7f3c2e000640 1 -- 172.21.15.47:0/64094335 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f3c24047640 con 0x7f3c380cbde0 2024-04-24T22:39:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-04-24T22:39:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.640+0000 7f3c3ca00640 1 -- 172.21.15.47:0/64094335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c380cbde0 msgr2=0x7f3c381e4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.640+0000 7f3c3ca00640 1 --2- 172.21.15.47:0/64094335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c380cbde0 0x7f3c381e4c00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3c2402a230 tx=0x7f3c240046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.640+0000 7f3c3ca00640 1 -- 172.21.15.47:0/64094335 shutdown_connections 2024-04-24T22:39:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.640+0000 7f3c3ca00640 1 --2- 172.21.15.47:0/64094335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3c380cc840 0x7f3c381e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.640+0000 7f3c3ca00640 1 --2- 172.21.15.47:0/64094335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c380cbde0 0x7f3c381e4c00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.640+0000 7f3c3ca00640 1 --2- 172.21.15.47:0/64094335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c38073380 0x7f3c38140b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.640+0000 7f3c3ca00640 1 -- 172.21.15.47:0/64094335 >> 172.21.15.47:0/64094335 conn(0x7f3c3806c940 msgr2=0x7f3c380c4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.640+0000 7f3c3ca00640 1 -- 172.21.15.47:0/64094335 shutdown_connections 2024-04-24T22:39:18.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:18.640+0000 7f3c3ca00640 1 -- 172.21.15.47:0/64094335 wait complete. 2024-04-24T22:39:18.644 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:18.642+0000 7fee40928040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:39:18.666 INFO:tasks.ceph:check_enable_crimson: False 2024-04-24T22:39:18.667 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-24T22:39:18.667 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:18.667 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-24T22:39:18.705 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:18.702+0000 7faa8bc14040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T22:39:18.741 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:18.741 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-24T22:39:18.795 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:18.794+0000 7fee40928040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:39:18.848 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:18.846+0000 7fee40928040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:39:18.864 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:18.864 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-24T22:39:18.898 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:18.895+0000 7faa8bc14040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:39:18.968 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:18.965+0000 7faa8bc14040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T22:39:18.977 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:39:18.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-24T22:39:19.057 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:19.057 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-24T22:39:19.081 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:19.082 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-24T22:39:19.099 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:19.097+0000 7fee40928040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:39:19.122 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:19.122 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-24T22:39:19.148 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:19.146+0000 7fee40928040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:39:19.171 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-24T22:39:19.171 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-24T22:39:19.234 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new aadbbff1-8bdf-473f-93ef-bc88982ffdce 0 2024-04-24T22:39:19.292 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:19.290+0000 7faa8bc14040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T22:39:19.348 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:19.345+0000 7faa8bc14040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T22:39:19.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.369+0000 7f78e8342640 1 Processor -- start 2024-04-24T22:39:19.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.370+0000 7f78e8342640 1 -- start start 2024-04-24T22:39:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.370+0000 7f78e8342640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78e0157a50 0x7f78e00734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.370+0000 7f78e8342640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e006c770 0x7f78e006b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.370+0000 7f78e8342640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78e015b050 con 0x7f78e0157e30 2024-04-24T22:39:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.370+0000 7f78e8342640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f78e015ded0 con 0x7f78e0157a50 2024-04-24T22:39:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.370+0000 7f78e8342640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f78e015b2f0 con 0x7f78e006c770 2024-04-24T22:39:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.371+0000 7f78e68df640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e006c770 0x7f78e006b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.371+0000 7f78e60de640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78e0157a50 0x7f78e00734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.371+0000 7f78e58dd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f78e0157e30 0x7f78e015e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33240/0 (socket says 172.21.15.113:33240) 2024-04-24T22:39:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.371+0000 7f78e58dd640 1 -- 172.21.15.113:0/3866745851 learned_addr learned my addr 172.21.15.113:0/3866745851 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T22:39:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.371+0000 7f78e60de640 1 -- 172.21.15.113:0/3866745851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e006c770 msgr2=0x7f78e006b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:19.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.371+0000 7f78e60de640 1 --2- 172.21.15.113:0/3866745851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e006c770 0x7f78e006b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.371+0000 7f78e60de640 1 -- 172.21.15.113:0/3866745851 >> v1:172.21.15.113:6789/0 conn(0x7f78e0157e30 legacy=0x7f78e015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.371+0000 7f78e50dc640 1 -- 172.21.15.113:0/3866745851 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3346481376 0 0) 0x7f78e015b050 con 0x7f78e0157e30 2024-04-24T22:39:19.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.371+0000 7f78e60de640 1 -- 172.21.15.113:0/3866745851 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e0071b20 con 0x7f78e0157a50 2024-04-24T22:39:19.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.371+0000 7f78e68df640 1 --2- 172.21.15.113:0/3866745851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e006c770 0x7f78e006b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:19.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.372+0000 7f78e60de640 1 --2- 172.21.15.113:0/3866745851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e0157a50 0x7f78e00734f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f78e015e050 tx=0x7f78dc02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578309ced9e72333 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:19.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.372+0000 7f78e50dc640 1 -- 172.21.15.113:0/3866745851 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc037070 con 0x7f78e0157a50 2024-04-24T22:39:19.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.372+0000 7f78e50dc640 1 -- 172.21.15.113:0/3866745851 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78dc032440 con 0x7f78e0157a50 2024-04-24T22:39:19.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.372+0000 7f78e50dc640 1 -- 172.21.15.113:0/3866745851 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc032810 con 0x7f78e0157a50 2024-04-24T22:39:19.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.373+0000 7f78e8342640 1 -- 172.21.15.113:0/3866745851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e0157a50 msgr2=0x7f78e00734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.373+0000 7f78e8342640 1 --2- 172.21.15.113:0/3866745851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e0157a50 0x7f78e00734f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f78e015e050 tx=0x7f78dc02a210 comp rx=0 tx=0).stop 2024-04-24T22:39:19.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.373+0000 7f78e8342640 1 -- 172.21.15.113:0/3866745851 shutdown_connections 2024-04-24T22:39:19.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.373+0000 7f78e8342640 1 --2- 172.21.15.113:0/3866745851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e006c770 0x7f78e006b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.373+0000 7f78e8342640 1 --2- 172.21.15.113:0/3866745851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e0157a50 0x7f78e00734f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.373+0000 7f78e8342640 1 -- 172.21.15.113:0/3866745851 >> 172.21.15.113:0/3866745851 conn(0x7f78e006f520 msgr2=0x7f78e006f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:19.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.373+0000 7f78e8342640 1 -- 172.21.15.113:0/3866745851 shutdown_connections 2024-04-24T22:39:19.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.373+0000 7f78e8342640 1 -- 172.21.15.113:0/3866745851 wait complete. 2024-04-24T22:39:19.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.374+0000 7f78e8342640 1 Processor -- start 2024-04-24T22:39:19.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.374+0000 7f78e8342640 1 -- start start 2024-04-24T22:39:19.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e8342640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e006c770 0x7f78e01e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e8342640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e0157a50 0x7f78e014a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e8342640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e0157e30 0x7f78e014aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e8342640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f78e015ea70 con 0x7f78e0157a50 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e8342640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f78e015db60 con 0x7f78e006c770 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e8342640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f78e015ba60 con 0x7f78e0157e30 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e60de640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e006c770 0x7f78e01e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e60de640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e006c770 0x7f78e01e49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53740/0 (socket says 172.21.15.113:53740) 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e60de640 1 -- 172.21.15.113:0/1221363405 learned_addr learned my addr 172.21.15.113:0/1221363405 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e68df640 1 --2- 172.21.15.113:0/1221363405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e0157e30 0x7f78e014aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e60de640 1 -- 172.21.15.113:0/1221363405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e0157e30 msgr2=0x7f78e014aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e60de640 1 --2- 172.21.15.113:0/1221363405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e0157e30 0x7f78e014aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e60de640 1 -- 172.21.15.113:0/1221363405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e0157a50 msgr2=0x7f78e014a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e60de640 1 --2- 172.21.15.113:0/1221363405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e0157a50 0x7f78e014a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e60de640 1 -- 172.21.15.113:0/1221363405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e01e6430 con 0x7f78e006c770 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e68df640 1 --2- 172.21.15.113:0/1221363405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e0157e30 0x7f78e014aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e60de640 1 --2- 172.21.15.113:0/1221363405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e006c770 0x7f78e01e49a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78dc000c00 tx=0x7f78dc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:19.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78d6ffd640 1 -- 172.21.15.113:0/1221363405 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc037040 con 0x7f78e006c770 2024-04-24T22:39:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78d6ffd640 1 -- 172.21.15.113:0/1221363405 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78dc042070 con 0x7f78e006c770 2024-04-24T22:39:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78d6ffd640 1 -- 172.21.15.113:0/1221363405 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc030880 con 0x7f78e006c770 2024-04-24T22:39:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e8342640 1 -- 172.21.15.113:0/1221363405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e01f12e0 con 0x7f78e006c770 2024-04-24T22:39:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.375+0000 7f78e8342640 1 -- 172.21.15.113:0/1221363405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e01f0520 con 0x7f78e006c770 2024-04-24T22:39:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.376+0000 7f78d6ffd640 1 -- 172.21.15.113:0/1221363405 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f78dc030a20 con 0x7f78e006c770 2024-04-24T22:39:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.376+0000 7f78d6ffd640 1 -- 172.21.15.113:0/1221363405 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f78dc047b20 con 0x7f78e006c770 2024-04-24T22:39:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.376+0000 7f78e8342640 1 -- 172.21.15.113:0/1221363405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e0156c00 con 0x7f78e006c770 2024-04-24T22:39:19.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.378+0000 7f78d6ffd640 1 -- 172.21.15.113:0/1221363405 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f78e0156c00 con 0x7f78e006c770 2024-04-24T22:39:19.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.449+0000 7f78e8342640 1 -- 172.21.15.113:0/1221363405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "aadbbff1-8bdf-473f-93ef-bc88982ffdce", "id": 0} v 0) -- 0x7f78e0073b90 con 0x7f78e006c770 2024-04-24T22:39:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.453+0000 7f78d6ffd640 1 -- 172.21.15.113:0/1221363405 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aadbbff1-8bdf-473f-93ef-bc88982ffdce", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f78dc047460 con 0x7f78e006c770 2024-04-24T22:39:19.455 INFO:teuthology.orchestra.run.smithi113.stdout:0 2024-04-24T22:39:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.454+0000 7f78e8342640 1 -- 172.21.15.113:0/1221363405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e006c770 msgr2=0x7f78e01e49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.454+0000 7f78e8342640 1 --2- 172.21.15.113:0/1221363405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e006c770 0x7f78e01e49a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f78dc000c00 tx=0x7f78dc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.454+0000 7f78e8342640 1 -- 172.21.15.113:0/1221363405 shutdown_connections 2024-04-24T22:39:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.454+0000 7f78e8342640 1 --2- 172.21.15.113:0/1221363405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78e0157e30 0x7f78e014aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.454+0000 7f78e8342640 1 --2- 172.21.15.113:0/1221363405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78e0157a50 0x7f78e014a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.454+0000 7f78e8342640 1 --2- 172.21.15.113:0/1221363405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78e006c770 0x7f78e01e49a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.454+0000 7f78e8342640 1 -- 172.21.15.113:0/1221363405 >> 172.21.15.113:0/1221363405 conn(0x7f78e006f520 msgr2=0x7f78e0154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:19.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.454+0000 7f78e8342640 1 -- 172.21.15.113:0/1221363405 shutdown_connections 2024-04-24T22:39:19.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.454+0000 7f78e8342640 1 -- 172.21.15.113:0/1221363405 wait complete. 2024-04-24T22:39:19.467 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 4ade4f53-014c-4118-8f91-a4e15071760e 1 2024-04-24T22:39:19.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.563+0000 7efeca438640 1 Processor -- start 2024-04-24T22:39:19.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.563+0000 7efeca438640 1 -- start start 2024-04-24T22:39:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efeca438640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efec40c9470 0x7efec4158e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efeca438640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec4062680 0x7efec4151e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efeca438640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec40c7170 con 0x7efec40c9090 2024-04-24T22:39:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efeca438640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efec40c8ca0 con 0x7efec40c9470 2024-04-24T22:39:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efeca438640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efec41593d0 con 0x7efec4062680 2024-04-24T22:39:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efec89d5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec4062680 0x7efec4151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efec37fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efec40c9470 0x7efec4158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efec89d5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec4062680 0x7efec4151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:44400/0 (socket says 172.21.15.113:44400) 2024-04-24T22:39:19.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efec89d5640 1 -- 172.21.15.113:0/3586508356 learned_addr learned my addr 172.21.15.113:0/3586508356 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efec37fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efec40c9470 0x7efec4158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53756/0 (socket says 172.21.15.113:53756) 2024-04-24T22:39:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efec37fe640 1 -- 172.21.15.113:0/3586508356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec4062680 msgr2=0x7efec4151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efec37fe640 1 --2- 172.21.15.113:0/3586508356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec4062680 0x7efec4151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efec37fe640 1 -- 172.21.15.113:0/3586508356 >> v1:172.21.15.47:6789/0 conn(0x7efec40c9090 legacy=0x7efec40c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efec37fe640 1 -- 172.21.15.113:0/3586508356 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec41583d0 con 0x7efec40c9470 2024-04-24T22:39:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.564+0000 7efec89d5640 1 --2- 172.21.15.113:0/3586508356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec4062680 0x7efec4151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.565+0000 7efec37fe640 1 --2- 172.21.15.113:0/3586508356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec40c9470 0x7efec4158e90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efeb80097a0 tx=0x7efeb802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f0124132c0e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.565+0000 7efec2ffd640 1 -- 172.21.15.113:0/3586508356 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8037070 con 0x7efec40c9470 2024-04-24T22:39:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.565+0000 7efec2ffd640 1 -- 172.21.15.113:0/3586508356 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb802ae40 con 0x7efec40c9470 2024-04-24T22:39:19.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.565+0000 7efec2ffd640 1 -- 172.21.15.113:0/3586508356 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8032660 con 0x7efec40c9470 2024-04-24T22:39:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.566+0000 7efeca438640 1 -- 172.21.15.113:0/3586508356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec40c9470 msgr2=0x7efec4158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.566+0000 7efeca438640 1 --2- 172.21.15.113:0/3586508356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec40c9470 0x7efec4158e90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efeb80097a0 tx=0x7efeb802a130 comp rx=0 tx=0).stop 2024-04-24T22:39:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.566+0000 7efeca438640 1 -- 172.21.15.113:0/3586508356 shutdown_connections 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.566+0000 7efeca438640 1 --2- 172.21.15.113:0/3586508356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec4062680 0x7efec4151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.566+0000 7efeca438640 1 --2- 172.21.15.113:0/3586508356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec40c9470 0x7efec4158e90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.566+0000 7efeca438640 1 -- 172.21.15.113:0/3586508356 >> 172.21.15.113:0/3586508356 conn(0x7efec406d850 msgr2=0x7efec406b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.566+0000 7efeca438640 1 -- 172.21.15.113:0/3586508356 shutdown_connections 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.566+0000 7efeca438640 1 -- 172.21.15.113:0/3586508356 wait complete. 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.567+0000 7efeca438640 1 Processor -- start 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.567+0000 7efeca438640 1 -- start start 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efeca438640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec4062680 0x7efec41502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efeca438640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec40c9090 0x7efec40cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efeca438640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40c9470 0x7efec414cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efeca438640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efec4158690 con 0x7efec40c9470 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efeca438640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efec40c7710 con 0x7efec4062680 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efeca438640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efec4158970 con 0x7efec40c9090 2024-04-24T22:39:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec4062680 0x7efec41502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec37fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec40c9090 0x7efec40cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec4062680 0x7efec41502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53760/0 (socket says 172.21.15.113:53760) 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec37fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec40c9090 0x7efec40cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:44414/0 (socket says 172.21.15.113:44414) 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec3fff640 1 -- 172.21.15.113:0/1864311396 learned_addr learned my addr 172.21.15.113:0/1864311396 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec37fe640 1 -- 172.21.15.113:0/1864311396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec4062680 msgr2=0x7efec41502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec37fe640 1 --2- 172.21.15.113:0/1864311396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec4062680 0x7efec41502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec37fe640 1 -- 172.21.15.113:0/1864311396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40c9470 msgr2=0x7efec414cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec37fe640 1 --2- 172.21.15.113:0/1864311396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40c9470 0x7efec414cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec37fe640 1 -- 172.21.15.113:0/1864311396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec41f14c0 con 0x7efec40c9090 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec3fff640 1 --2- 172.21.15.113:0/1864311396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec4062680 0x7efec41502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec37fe640 1 --2- 172.21.15.113:0/1864311396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec40c9090 0x7efec40cc920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efeb8031000 tx=0x7efeb80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec0ff9640 1 -- 172.21.15.113:0/1864311396 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8041020 con 0x7efec40c9090 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec0ff9640 1 -- 172.21.15.113:0/1864311396 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb803c670 con 0x7efec40c9090 2024-04-24T22:39:19.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efeca438640 1 -- 172.21.15.113:0/1864311396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec40cbf10 con 0x7efec40c9090 2024-04-24T22:39:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efec0ff9640 1 -- 172.21.15.113:0/1864311396 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8037040 con 0x7efec40c9090 2024-04-24T22:39:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.568+0000 7efeca438640 1 -- 172.21.15.113:0/1864311396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efec40cb390 con 0x7efec40c9090 2024-04-24T22:39:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.569+0000 7efec0ff9640 1 -- 172.21.15.113:0/1864311396 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efeb8044020 con 0x7efec40c9090 2024-04-24T22:39:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.569+0000 7efec0ff9640 1 -- 172.21.15.113:0/1864311396 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7efeb803dd50 con 0x7efec40c9090 2024-04-24T22:39:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.569+0000 7efeca438640 1 -- 172.21.15.113:0/1864311396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec4069810 con 0x7efec40c9090 2024-04-24T22:39:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.571+0000 7efec0ff9640 1 -- 172.21.15.113:0/1864311396 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7efeb804b030 con 0x7efec40c9090 2024-04-24T22:39:19.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.644+0000 7efeca438640 1 -- 172.21.15.113:0/1864311396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4ade4f53-014c-4118-8f91-a4e15071760e", "id": 1} v 0) -- 0x7efec4073b90 con 0x7efec40c9090 2024-04-24T22:39:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.649+0000 7efec0ff9640 1 -- 172.21.15.113:0/1864311396 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4ade4f53-014c-4118-8f91-a4e15071760e", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7efec41f14c0 con 0x7efec40c9090 2024-04-24T22:39:19.651 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-24T22:39:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.650+0000 7efeca438640 1 -- 172.21.15.113:0/1864311396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec40c9090 msgr2=0x7efec40cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.650+0000 7efeca438640 1 --2- 172.21.15.113:0/1864311396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec40c9090 0x7efec40cc920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efeb8031000 tx=0x7efeb80046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.650+0000 7efeca438640 1 -- 172.21.15.113:0/1864311396 shutdown_connections 2024-04-24T22:39:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.650+0000 7efeca438640 1 --2- 172.21.15.113:0/1864311396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efec40c9470 0x7efec414cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.650+0000 7efeca438640 1 --2- 172.21.15.113:0/1864311396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efec40c9090 0x7efec40cc920 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.650+0000 7efeca438640 1 --2- 172.21.15.113:0/1864311396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efec4062680 0x7efec41502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.650+0000 7efeca438640 1 -- 172.21.15.113:0/1864311396 >> 172.21.15.113:0/1864311396 conn(0x7efec406d850 msgr2=0x7efec40c9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:19.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.650+0000 7efeca438640 1 -- 172.21.15.113:0/1864311396 shutdown_connections 2024-04-24T22:39:19.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.650+0000 7efeca438640 1 -- 172.21.15.113:0/1864311396 wait complete. 2024-04-24T22:39:19.662 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 252f4784-03e2-4b28-bb96-9ce9c33a40cd 2 2024-04-24T22:39:19.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.757+0000 7fdbe0df9640 1 Processor -- start 2024-04-24T22:39:19.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.757+0000 7fdbe0df9640 1 -- start start 2024-04-24T22:39:19.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbe0df9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdbdc154570 0x7fdbdc0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbe0df9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdbdc0ca1c0 0x7fdbdc15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbe0df9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc15a520 0x7fdbdc15e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbe0df9640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdbdc15a9b0 con 0x7fdbdc154570 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbe0df9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdbdc15ac90 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbe0df9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc15af70 con 0x7fdbdc15a520 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbdad9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc15a520 0x7fdbdc15e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbd3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdbdc0ca1c0 0x7fdbdc15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbd3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdbdc0ca1c0 0x7fdbdc15e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53770/0 (socket says 172.21.15.113:53770) 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbdad9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc15a520 0x7fdbdc15e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:44424/0 (socket says 172.21.15.113:44424) 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbd3fff640 1 -- 172.21.15.113:0/1314017383 learned_addr learned my addr 172.21.15.113:0/1314017383 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbdad9d640 1 -- 172.21.15.113:0/1314017383 >> v2:172.21.15.113:3300/0 conn(0x7fdbdc0ca1c0 msgr2=0x7fdbdc15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbdad9d640 1 --2- 172.21.15.113:0/1314017383 >> v2:172.21.15.113:3300/0 conn(0x7fdbdc0ca1c0 0x7fdbdc15e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbdad9d640 1 -- 172.21.15.113:0/1314017383 >> v2:172.21.15.47:3300/0 conn(0x7fdbdc154570 msgr2=0x7fdbdc0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbdad9d640 1 --2- 172.21.15.113:0/1314017383 >> v2:172.21.15.47:3300/0 conn(0x7fdbdc154570 0x7fdbdc0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.758+0000 7fdbdad9d640 1 -- 172.21.15.113:0/1314017383 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc15b250 con 0x7fdbdc15a520 2024-04-24T22:39:19.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.759+0000 7fdbdad9d640 1 --2- 172.21.15.113:0/1314017383 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc15a520 0x7fdbdc15e710 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc15aea0 tx=0x7fdbd402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ea593675666017 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:19.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.759+0000 7fdbd9d9b640 1 -- 172.21.15.113:0/1314017383 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4031600 con 0x7fdbdc15a520 2024-04-24T22:39:19.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.759+0000 7fdbd9d9b640 1 -- 172.21.15.113:0/1314017383 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd40317a0 con 0x7fdbdc15a520 2024-04-24T22:39:19.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.759+0000 7fdbd9d9b640 1 -- 172.21.15.113:0/1314017383 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4031b70 con 0x7fdbdc15a520 2024-04-24T22:39:19.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.760+0000 7fdbe0df9640 1 -- 172.21.15.113:0/1314017383 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc15a520 msgr2=0x7fdbdc15e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.760+0000 7fdbe0df9640 1 --2- 172.21.15.113:0/1314017383 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc15a520 0x7fdbdc15e710 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc15aea0 tx=0x7fdbd402a5d0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.760+0000 7fdbe0df9640 1 -- 172.21.15.113:0/1314017383 shutdown_connections 2024-04-24T22:39:19.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.760+0000 7fdbe0df9640 1 --2- 172.21.15.113:0/1314017383 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc15a520 0x7fdbdc15e710 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.760+0000 7fdbe0df9640 1 --2- 172.21.15.113:0/1314017383 >> v2:172.21.15.113:3300/0 conn(0x7fdbdc0ca1c0 0x7fdbdc15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.760+0000 7fdbe0df9640 1 --2- 172.21.15.113:0/1314017383 >> v2:172.21.15.47:3300/0 conn(0x7fdbdc154570 0x7fdbdc0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.760+0000 7fdbe0df9640 1 -- 172.21.15.113:0/1314017383 >> 172.21.15.113:0/1314017383 conn(0x7fdbdc073420 msgr2=0x7fdbdc070730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:19.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.760+0000 7fdbe0df9640 1 -- 172.21.15.113:0/1314017383 shutdown_connections 2024-04-24T22:39:19.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.760+0000 7fdbe0df9640 1 -- 172.21.15.113:0/1314017383 wait complete. 2024-04-24T22:39:19.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.761+0000 7fdbe0df9640 1 Processor -- start 2024-04-24T22:39:19.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.761+0000 7fdbe0df9640 1 -- start start 2024-04-24T22:39:19.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbe0df9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc0ca1c0 0x7fdbdc152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbe0df9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbdc154570 0x7fdbdc15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbe0df9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc15d840 0x7fdbdc1442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbe0df9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc0c7f50 con 0x7fdbdc154570 2024-04-24T22:39:19.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbe0df9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc154950 con 0x7fdbdc15d840 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbe0df9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc0c80d0 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbda59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc0ca1c0 0x7fdbdc152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbdad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc15d840 0x7fdbdc1442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbda59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc0ca1c0 0x7fdbdc152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:44440/0 (socket says 172.21.15.113:44440) 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbda59c640 1 -- 172.21.15.113:0/635620930 learned_addr learned my addr 172.21.15.113:0/635620930 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbda59c640 1 -- 172.21.15.113:0/635620930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc15d840 msgr2=0x7fdbdc1442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbda59c640 1 --2- 172.21.15.113:0/635620930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc15d840 0x7fdbdc1442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbda59c640 1 -- 172.21.15.113:0/635620930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbdc154570 msgr2=0x7fdbdc15c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbda59c640 1 --2- 172.21.15.113:0/635620930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbdc154570 0x7fdbdc15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbda59c640 1 -- 172.21.15.113:0/635620930 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc1f0de0 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbdad9d640 1 --2- 172.21.15.113:0/635620930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc15d840 0x7fdbdc1442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbda59c640 1 --2- 172.21.15.113:0/635620930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc0ca1c0 0x7fdbdc152fb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00b4d0 tx=0x7fdbcc00b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.762+0000 7fdbd2ffd640 1 -- 172.21.15.113:0/635620930 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc014070 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.763+0000 7fdbd2ffd640 1 -- 172.21.15.113:0/635620930 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc010870 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.763+0000 7fdbe0df9640 1 -- 172.21.15.113:0/635620930 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc1f10c0 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.763+0000 7fdbd2ffd640 1 -- 172.21.15.113:0/635620930 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc010c60 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.763+0000 7fdbe0df9640 1 -- 172.21.15.113:0/635620930 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc15bc10 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.763+0000 7fdbd2ffd640 1 -- 172.21.15.113:0/635620930 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdbcc017070 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.763+0000 7fdbd2ffd640 1 -- 172.21.15.113:0/635620930 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fdbcc00f830 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.763+0000 7fdbe0df9640 1 -- 172.21.15.113:0/635620930 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc073b90 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.766+0000 7fdbd2ffd640 1 -- 172.21.15.113:0/635620930 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdbcc00f410 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.838+0000 7fdbe0df9640 1 -- 172.21.15.113:0/635620930 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "252f4784-03e2-4b28-bb96-9ce9c33a40cd", "id": 2} v 0) -- 0x7fdbdc0c3030 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.843+0000 7fdbd2ffd640 1 -- 172.21.15.113:0/635620930 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "252f4784-03e2-4b28-bb96-9ce9c33a40cd", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fdbcc00fb10 con 0x7fdbdc0ca1c0 2024-04-24T22:39:19.845 INFO:teuthology.orchestra.run.smithi113.stdout:2 2024-04-24T22:39:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.844+0000 7fdbe0df9640 1 -- 172.21.15.113:0/635620930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc0ca1c0 msgr2=0x7fdbdc152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.844+0000 7fdbe0df9640 1 --2- 172.21.15.113:0/635620930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc0ca1c0 0x7fdbdc152fb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00b4d0 tx=0x7fdbcc00b990 comp rx=0 tx=0).stop 2024-04-24T22:39:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.844+0000 7fdbe0df9640 1 -- 172.21.15.113:0/635620930 shutdown_connections 2024-04-24T22:39:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.844+0000 7fdbe0df9640 1 --2- 172.21.15.113:0/635620930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc15d840 0x7fdbdc1442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.844+0000 7fdbe0df9640 1 --2- 172.21.15.113:0/635620930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbdc154570 0x7fdbdc15c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.844+0000 7fdbe0df9640 1 --2- 172.21.15.113:0/635620930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbdc0ca1c0 0x7fdbdc152fb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.844+0000 7fdbe0df9640 1 -- 172.21.15.113:0/635620930 >> 172.21.15.113:0/635620930 conn(0x7fdbdc073420 msgr2=0x7fdbdc0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:19.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.844+0000 7fdbe0df9640 1 -- 172.21.15.113:0/635620930 shutdown_connections 2024-04-24T22:39:19.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.844+0000 7fdbe0df9640 1 -- 172.21.15.113:0/635620930 wait complete. 2024-04-24T22:39:19.856 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new ee980f09-6a59-4d99-818e-431e9e755849 3 2024-04-24T22:39:19.864 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:19.862+0000 7fee40928040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:39:19.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.951+0000 7f77de709640 1 Processor -- start 2024-04-24T22:39:19.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.951+0000 7f77de709640 1 -- start start 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77de709640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77d8154570 0x7f77d80c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77de709640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d815e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77de709640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77d815a520 0x7f77d815e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77de709640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f77d815a9b0 con 0x7f77d815a520 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77de709640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f77d815ac90 con 0x7f77d8154570 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77de709640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f77d815af70 con 0x7f77d80ca1c0 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77d8154570 0x7f77d80c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d77fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d815e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77d8154570 0x7f77d80c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53786/0 (socket says 172.21.15.113:53786) 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d77fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d815e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:44454/0 (socket says 172.21.15.113:44454) 2024-04-24T22:39:19.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d7fff640 1 -- 172.21.15.113:0/658692103 learned_addr learned my addr 172.21.15.113:0/658692103 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:19.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d7fff640 1 -- 172.21.15.113:0/658692103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 msgr2=0x7f77d815e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:19.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d7fff640 1 --2- 172.21.15.113:0/658692103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d815e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d7fff640 1 -- 172.21.15.113:0/658692103 >> v2:172.21.15.47:3300/0 conn(0x7f77d815a520 msgr2=0x7f77d815e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:19.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d7fff640 1 --2- 172.21.15.113:0/658692103 >> v2:172.21.15.47:3300/0 conn(0x7f77d815a520 0x7f77d815e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d7fff640 1 -- 172.21.15.113:0/658692103 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d815b250 con 0x7f77d8154570 2024-04-24T22:39:19.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.952+0000 7f77d77fe640 1 --2- 172.21.15.113:0/658692103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d815e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:19.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.953+0000 7f77d7fff640 1 --2- 172.21.15.113:0/658692103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 0x7f77d80c9c80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f77d8071680 tx=0x7f77cc02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edf844f9d513a480 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:19.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.953+0000 7f77d6ffd640 1 -- 172.21.15.113:0/658692103 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc037070 con 0x7f77d8154570 2024-04-24T22:39:19.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.953+0000 7f77d6ffd640 1 -- 172.21.15.113:0/658692103 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77cc032440 con 0x7f77d8154570 2024-04-24T22:39:19.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.953+0000 7f77d6ffd640 1 -- 172.21.15.113:0/658692103 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc032810 con 0x7f77d8154570 2024-04-24T22:39:19.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.954+0000 7f77de709640 1 -- 172.21.15.113:0/658692103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 msgr2=0x7f77d80c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.954+0000 7f77de709640 1 --2- 172.21.15.113:0/658692103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 0x7f77d80c9c80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f77d8071680 tx=0x7f77cc02a1e0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.954+0000 7f77de709640 1 -- 172.21.15.113:0/658692103 shutdown_connections 2024-04-24T22:39:19.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.954+0000 7f77de709640 1 --2- 172.21.15.113:0/658692103 >> v2:172.21.15.47:3300/0 conn(0x7f77d815a520 0x7f77d815e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.954+0000 7f77de709640 1 --2- 172.21.15.113:0/658692103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d815e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.954+0000 7f77de709640 1 --2- 172.21.15.113:0/658692103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 0x7f77d80c9c80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.954+0000 7f77de709640 1 -- 172.21.15.113:0/658692103 >> 172.21.15.113:0/658692103 conn(0x7f77d8073420 msgr2=0x7f77d8070730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.954+0000 7f77de709640 1 -- 172.21.15.113:0/658692103 shutdown_connections 2024-04-24T22:39:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.954+0000 7f77de709640 1 -- 172.21.15.113:0/658692103 wait complete. 2024-04-24T22:39:19.958 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:19.955+0000 7fee40928040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:39:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.955+0000 7f77de709640 1 Processor -- start 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.955+0000 7f77de709640 1 -- start start 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77de709640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d81f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77de709640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 0x7f77d81f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77de709640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d81f0190 0x7f77d81526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77de709640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80c7f50 con 0x7f77d81f0190 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77de709640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8154950 con 0x7f77d8154570 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77de709640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f77d80c80d0 con 0x7f77d80ca1c0 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d81f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 0x7f77d81f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d81f4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:44462/0 (socket says 172.21.15.113:44462) 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d7fff640 1 -- 172.21.15.113:0/2536193405 learned_addr learned my addr 172.21.15.113:0/2536193405 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 0x7f77d81f0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53800/0 (socket says 172.21.15.113:53800) 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d7fff640 1 -- 172.21.15.113:0/2536193405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 msgr2=0x7f77d81f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77dcca6640 1 --2- 172.21.15.113:0/2536193405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d81f0190 0x7f77d81526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d7fff640 1 --2- 172.21.15.113:0/2536193405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 0x7f77d81f0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d7fff640 1 -- 172.21.15.113:0/2536193405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d81f0190 msgr2=0x7f77d81526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d7fff640 1 --2- 172.21.15.113:0/2536193405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d81f0190 0x7f77d81526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d7fff640 1 -- 172.21.15.113:0/2536193405 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d81f45c0 con 0x7f77d80ca1c0 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d77fe640 1 --2- 172.21.15.113:0/2536193405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 0x7f77d81f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.956+0000 7f77d7fff640 1 --2- 172.21.15.113:0/2536193405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d81f4080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f77cc02a220 tx=0x7f77cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.957+0000 7f77d4ff9640 1 -- 172.21.15.113:0/2536193405 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc037040 con 0x7f77d80ca1c0 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.957+0000 7f77d4ff9640 1 -- 172.21.15.113:0/2536193405 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77cc042070 con 0x7f77d80ca1c0 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.957+0000 7f77de709640 1 -- 172.21.15.113:0/2536193405 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d8150250 con 0x7f77d80ca1c0 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.957+0000 7f77d4ff9640 1 -- 172.21.15.113:0/2536193405 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc030860 con 0x7f77d80ca1c0 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.957+0000 7f77de709640 1 -- 172.21.15.113:0/2536193405 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d8150810 con 0x7f77d80ca1c0 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.957+0000 7f77d4ff9640 1 -- 172.21.15.113:0/2536193405 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77cc030a00 con 0x7f77d80ca1c0 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.957+0000 7f77d4ff9640 1 -- 172.21.15.113:0/2536193405 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f77cc03b430 con 0x7f77d80ca1c0 2024-04-24T22:39:19.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.958+0000 7f77de709640 1 -- 172.21.15.113:0/2536193405 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d8073b90 con 0x7f77d80ca1c0 2024-04-24T22:39:19.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:19.960+0000 7f77d4ff9640 1 -- 172.21.15.113:0/2536193405 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f77cc047460 con 0x7f77d80ca1c0 2024-04-24T22:39:20.001 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:20.000+0000 7fee40928040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:39:20.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.032+0000 7f77de709640 1 -- 172.21.15.113:0/2536193405 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ee980f09-6a59-4d99-818e-431e9e755849", "id": 3} v 0) -- 0x7f77d80c3030 con 0x7f77d80ca1c0 2024-04-24T22:39:20.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.037+0000 7f77d4ff9640 1 -- 172.21.15.113:0/2536193405 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ee980f09-6a59-4d99-818e-431e9e755849", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f77cc047700 con 0x7f77d80ca1c0 2024-04-24T22:39:20.039 INFO:teuthology.orchestra.run.smithi113.stdout:3 2024-04-24T22:39:20.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.037+0000 7f77de709640 1 -- 172.21.15.113:0/2536193405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 msgr2=0x7f77d81f4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.037+0000 7f77de709640 1 --2- 172.21.15.113:0/2536193405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d81f4080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f77cc02a220 tx=0x7f77cc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.037+0000 7f77de709640 1 -- 172.21.15.113:0/2536193405 shutdown_connections 2024-04-24T22:39:20.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.037+0000 7f77de709640 1 --2- 172.21.15.113:0/2536193405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d81f0190 0x7f77d81526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.037+0000 7f77de709640 1 --2- 172.21.15.113:0/2536193405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77d8154570 0x7f77d81f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.037+0000 7f77de709640 1 --2- 172.21.15.113:0/2536193405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77d80ca1c0 0x7f77d81f4080 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.037+0000 7f77de709640 1 -- 172.21.15.113:0/2536193405 >> 172.21.15.113:0/2536193405 conn(0x7f77d8073420 msgr2=0x7f77d80c9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:20.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.038+0000 7f77de709640 1 -- 172.21.15.113:0/2536193405 shutdown_connections 2024-04-24T22:39:20.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.038+0000 7f77de709640 1 -- 172.21.15.113:0/2536193405 wait complete. 2024-04-24T22:39:20.050 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 9c4356d4-c36e-480c-bb3d-887f722e946c 4 2024-04-24T22:39:20.176 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:20.174+0000 7fee40928040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:39:20.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.186+0000 7efd6f5d5640 1 Processor -- start 2024-04-24T22:39:20.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.186+0000 7efd6f5d5640 1 -- start start 2024-04-24T22:39:20.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.186+0000 7efd6f5d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd680cc840 0x7efd680ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.186+0000 7efd6f5d5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efd68073380 0x7efd680ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd6f5d5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd6815b240 con 0x7efd680cb6d0 2024-04-24T22:39:20.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd6f5d5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7efd6815daa0 con 0x7efd68073380 2024-04-24T22:39:20.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd6f5d5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efd6815dcf0 con 0x7efd680cc840 2024-04-24T22:39:20.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd6d371640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd680cc840 0x7efd680ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd6db72640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efd680cb6d0 0x7efd680cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33244/0 (socket says 172.21.15.113:33244) 2024-04-24T22:39:20.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd6db72640 1 -- 172.21.15.113:0/1589333972 learned_addr learned my addr 172.21.15.113:0/1589333972 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T22:39:20.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd6d371640 1 -- 172.21.15.113:0/1589333972 >> v2:172.21.15.47:3300/0 conn(0x7efd68073380 msgr2=0x7efd680ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:20.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd6d371640 1 --2- 172.21.15.113:0/1589333972 >> v2:172.21.15.47:3300/0 conn(0x7efd68073380 0x7efd680ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd6d371640 1 -- 172.21.15.113:0/1589333972 >> v1:172.21.15.113:6789/0 conn(0x7efd680cb6d0 legacy=0x7efd680cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd5ffff640 1 -- 172.21.15.113:0/1589333972 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2890740229 0 0) 0x7efd6815b240 con 0x7efd680cb6d0 2024-04-24T22:39:20.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.187+0000 7efd6d371640 1 -- 172.21.15.113:0/1589333972 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6815dfd0 con 0x7efd680cc840 2024-04-24T22:39:20.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.188+0000 7efd6d371640 1 --2- 172.21.15.113:0/1589333972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd680cc840 0x7efd680ca7d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd68154d70 tx=0x7efd5802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45175a716c0986c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:20.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.188+0000 7efd5ffff640 1 -- 172.21.15.113:0/1589333972 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58037070 con 0x7efd680cc840 2024-04-24T22:39:20.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.188+0000 7efd5ffff640 1 -- 172.21.15.113:0/1589333972 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd58032440 con 0x7efd680cc840 2024-04-24T22:39:20.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.188+0000 7efd5ffff640 1 -- 172.21.15.113:0/1589333972 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58032810 con 0x7efd680cc840 2024-04-24T22:39:20.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.188+0000 7efd6f5d5640 1 -- 172.21.15.113:0/1589333972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd680cc840 msgr2=0x7efd680ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.188+0000 7efd6f5d5640 1 --2- 172.21.15.113:0/1589333972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd680cc840 0x7efd680ca7d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd68154d70 tx=0x7efd5802a1a0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.188+0000 7efd6f5d5640 1 -- 172.21.15.113:0/1589333972 shutdown_connections 2024-04-24T22:39:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.188+0000 7efd6f5d5640 1 --2- 172.21.15.113:0/1589333972 >> v2:172.21.15.47:3300/0 conn(0x7efd68073380 0x7efd680ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.188+0000 7efd6f5d5640 1 --2- 172.21.15.113:0/1589333972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd680cc840 0x7efd680ca7d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.188+0000 7efd6f5d5640 1 -- 172.21.15.113:0/1589333972 >> 172.21.15.113:0/1589333972 conn(0x7efd6806c940 msgr2=0x7efd68072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.189+0000 7efd6f5d5640 1 -- 172.21.15.113:0/1589333972 shutdown_connections 2024-04-24T22:39:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.189+0000 7efd6f5d5640 1 -- 172.21.15.113:0/1589333972 wait complete. 2024-04-24T22:39:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6f5d5640 1 Processor -- start 2024-04-24T22:39:20.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6f5d5640 1 -- start start 2024-04-24T22:39:20.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6f5d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd68073380 0x7efd6815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6f5d5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd680cb6d0 0x7efd680ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6f5d5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd680cc840 0x7efd680ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6f5d5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efd6815e870 con 0x7efd680cb6d0 2024-04-24T22:39:20.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6f5d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd6815e590 con 0x7efd68073380 2024-04-24T22:39:20.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6f5d5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efd6815af30 con 0x7efd680cc840 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6d371640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd68073380 0x7efd6815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6d371640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd68073380 0x7efd6815cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53818/0 (socket says 172.21.15.113:53818) 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6db72640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd680cc840 0x7efd680ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.190+0000 7efd6d371640 1 -- 172.21.15.113:0/3028161128 learned_addr learned my addr 172.21.15.113:0/3028161128 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd6d371640 1 -- 172.21.15.113:0/3028161128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd680cc840 msgr2=0x7efd680ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd6d371640 1 --2- 172.21.15.113:0/3028161128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd680cc840 0x7efd680ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd6d371640 1 -- 172.21.15.113:0/3028161128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd680cb6d0 msgr2=0x7efd680ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd6d371640 1 --2- 172.21.15.113:0/3028161128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd680cb6d0 0x7efd680ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd6d371640 1 -- 172.21.15.113:0/3028161128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd680c9d30 con 0x7efd68073380 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd6db72640 1 --2- 172.21.15.113:0/3028161128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd680cc840 0x7efd680ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd6d371640 1 --2- 172.21.15.113:0/3028161128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd68073380 0x7efd6815cb40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efd5802a790 tx=0x7efd580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd5dffb640 1 -- 172.21.15.113:0/3028161128 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58041070 con 0x7efd68073380 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd5dffb640 1 -- 172.21.15.113:0/3028161128 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5803c950 con 0x7efd68073380 2024-04-24T22:39:20.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd6f5d5640 1 -- 172.21.15.113:0/3028161128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd681e5320 con 0x7efd68073380 2024-04-24T22:39:20.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd5dffb640 1 -- 172.21.15.113:0/3028161128 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58037040 con 0x7efd68073380 2024-04-24T22:39:20.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd6f5d5640 1 -- 172.21.15.113:0/3028161128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd681e58e0 con 0x7efd68073380 2024-04-24T22:39:20.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd5dffb640 1 -- 172.21.15.113:0/3028161128 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efd58044070 con 0x7efd68073380 2024-04-24T22:39:20.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.191+0000 7efd5dffb640 1 -- 172.21.15.113:0/3028161128 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7efd5803d3f0 con 0x7efd68073380 2024-04-24T22:39:20.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.192+0000 7efd6f5d5640 1 -- 172.21.15.113:0/3028161128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd68073b90 con 0x7efd68073380 2024-04-24T22:39:20.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.196+0000 7efd5dffb640 1 -- 172.21.15.113:0/3028161128 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7efd5803cd40 con 0x7efd68073380 2024-04-24T22:39:20.228 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:20.226+0000 7fee40928040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:39:20.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.268+0000 7efd6f5d5640 1 -- 172.21.15.113:0/3028161128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9c4356d4-c36e-480c-bb3d-887f722e946c", "id": 4} v 0) -- 0x7efd680cdd10 con 0x7efd68073380 2024-04-24T22:39:20.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.273+0000 7efd5dffb640 1 -- 172.21.15.113:0/3028161128 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c4356d4-c36e-480c-bb3d-887f722e946c", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7efd58046450 con 0x7efd68073380 2024-04-24T22:39:20.275 INFO:teuthology.orchestra.run.smithi113.stdout:4 2024-04-24T22:39:20.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.274+0000 7efd6f5d5640 1 -- 172.21.15.113:0/3028161128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd68073380 msgr2=0x7efd6815cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.274+0000 7efd6f5d5640 1 --2- 172.21.15.113:0/3028161128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd68073380 0x7efd6815cb40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efd5802a790 tx=0x7efd580046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.275+0000 7efd6f5d5640 1 -- 172.21.15.113:0/3028161128 shutdown_connections 2024-04-24T22:39:20.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.275+0000 7efd6f5d5640 1 --2- 172.21.15.113:0/3028161128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd680cc840 0x7efd680ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.275+0000 7efd6f5d5640 1 --2- 172.21.15.113:0/3028161128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd680cb6d0 0x7efd680ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.275+0000 7efd6f5d5640 1 --2- 172.21.15.113:0/3028161128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd68073380 0x7efd6815cb40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.275+0000 7efd6f5d5640 1 -- 172.21.15.113:0/3028161128 >> 172.21.15.113:0/3028161128 conn(0x7efd6806c940 msgr2=0x7efd680c5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:20.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.275+0000 7efd6f5d5640 1 -- 172.21.15.113:0/3028161128 shutdown_connections 2024-04-24T22:39:20.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.275+0000 7efd6f5d5640 1 -- 172.21.15.113:0/3028161128 wait complete. 2024-04-24T22:39:20.283 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:20.281+0000 7fee40928040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:39:20.287 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 55580504-e9dc-47e0-841b-3a022c2b3328 5 2024-04-24T22:39:20.358 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:20.355+0000 7faa8bc14040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T22:39:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.381+0000 7f49b808c640 1 Processor -- start 2024-04-24T22:39:20.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.381+0000 7f49b808c640 1 -- start start 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b808c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49b00cc840 0x7f49b00ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b808c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49b0073380 0x7f49b00ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b808c640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b015b240 con 0x7f49b00cb6d0 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b808c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f49b015daa0 con 0x7f49b0073380 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b808c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f49b015dcf0 con 0x7f49b00cc840 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b5e28640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49b00cc840 0x7f49b00ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b5e28640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49b00cc840 0x7f49b00ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53830/0 (socket says 172.21.15.113:53830) 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b5e28640 1 -- 172.21.15.113:0/3840067829 learned_addr learned my addr 172.21.15.113:0/3840067829 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b5e28640 1 -- 172.21.15.113:0/3840067829 >> v2:172.21.15.47:3300/0 conn(0x7f49b0073380 msgr2=0x7f49b00ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b5e28640 1 --2- 172.21.15.113:0/3840067829 >> v2:172.21.15.47:3300/0 conn(0x7f49b0073380 0x7f49b00ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b5e28640 1 -- 172.21.15.113:0/3840067829 >> v1:172.21.15.47:6789/0 conn(0x7f49b00cb6d0 legacy=0x7f49b00cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.382+0000 7f49b5e28640 1 -- 172.21.15.113:0/3840067829 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b015dfd0 con 0x7f49b00cc840 2024-04-24T22:39:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.383+0000 7f49b5e28640 1 --2- 172.21.15.113:0/3840067829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cc840 0x7f49b00ca7d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49b0154d70 tx=0x7f49ac02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e584298ef97ec01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.383+0000 7f49b4e26640 1 -- 172.21.15.113:0/3840067829 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac037070 con 0x7f49b00cc840 2024-04-24T22:39:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.383+0000 7f49b4e26640 1 -- 172.21.15.113:0/3840067829 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ac032440 con 0x7f49b00cc840 2024-04-24T22:39:20.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.383+0000 7f49b4e26640 1 -- 172.21.15.113:0/3840067829 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac032810 con 0x7f49b00cc840 2024-04-24T22:39:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.383+0000 7f49b808c640 1 -- 172.21.15.113:0/3840067829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cc840 msgr2=0x7f49b00ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.383+0000 7f49b808c640 1 --2- 172.21.15.113:0/3840067829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cc840 0x7f49b00ca7d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49b0154d70 tx=0x7f49ac02a230 comp rx=0 tx=0).stop 2024-04-24T22:39:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.384+0000 7f49b808c640 1 -- 172.21.15.113:0/3840067829 shutdown_connections 2024-04-24T22:39:20.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.384+0000 7f49b808c640 1 --2- 172.21.15.113:0/3840067829 >> v2:172.21.15.47:3300/0 conn(0x7f49b0073380 0x7f49b00ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.384+0000 7f49b808c640 1 --2- 172.21.15.113:0/3840067829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cc840 0x7f49b00ca7d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.384+0000 7f49b808c640 1 -- 172.21.15.113:0/3840067829 >> 172.21.15.113:0/3840067829 conn(0x7f49b006c940 msgr2=0x7f49b0072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.384+0000 7f49b808c640 1 -- 172.21.15.113:0/3840067829 shutdown_connections 2024-04-24T22:39:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.384+0000 7f49b808c640 1 -- 172.21.15.113:0/3840067829 wait complete. 2024-04-24T22:39:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.385+0000 7f49b808c640 1 Processor -- start 2024-04-24T22:39:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.385+0000 7f49b808c640 1 -- start start 2024-04-24T22:39:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.385+0000 7f49b808c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0073380 0x7f49b00cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.385+0000 7f49b808c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cb6d0 0x7f49b01e6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.385+0000 7f49b808c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b00cc840 0x7f49b0159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.385+0000 7f49b808c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f49b015e870 con 0x7f49b0073380 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.385+0000 7f49b808c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49b015e590 con 0x7f49b00cb6d0 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.385+0000 7f49b808c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f49b015af30 con 0x7f49b00cc840 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b5627640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cb6d0 0x7f49b01e6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b6629640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b00cc840 0x7f49b0159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b5627640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cb6d0 0x7f49b01e6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53834/0 (socket says 172.21.15.113:53834) 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b5627640 1 -- 172.21.15.113:0/2490878044 learned_addr learned my addr 172.21.15.113:0/2490878044 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b6629640 1 -- 172.21.15.113:0/2490878044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cb6d0 msgr2=0x7f49b01e6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b6629640 1 --2- 172.21.15.113:0/2490878044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cb6d0 0x7f49b01e6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b6629640 1 -- 172.21.15.113:0/2490878044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0073380 msgr2=0x7f49b00cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b6629640 1 --2- 172.21.15.113:0/2490878044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0073380 0x7f49b00cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b6629640 1 -- 172.21.15.113:0/2490878044 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b01e66d0 con 0x7f49b00cc840 2024-04-24T22:39:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b5627640 1 --2- 172.21.15.113:0/2490878044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cb6d0 0x7f49b01e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b6629640 1 --2- 172.21.15.113:0/2490878044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b00cc840 0x7f49b0159150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f49a800c7b0 tx=0x7f49a800cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49a67fc640 1 -- 172.21.15.113:0/2490878044 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8017070 con 0x7f49b00cc840 2024-04-24T22:39:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49a67fc640 1 -- 172.21.15.113:0/2490878044 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a8003b90 con 0x7f49b00cc840 2024-04-24T22:39:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b808c640 1 -- 172.21.15.113:0/2490878044 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b0157110 con 0x7f49b00cc840 2024-04-24T22:39:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49a67fc640 1 -- 172.21.15.113:0/2490878044 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8005070 con 0x7f49b00cc840 2024-04-24T22:39:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49b808c640 1 -- 172.21.15.113:0/2490878044 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b01e4560 con 0x7f49b00cc840 2024-04-24T22:39:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49a67fc640 1 -- 172.21.15.113:0/2490878044 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f49a801a070 con 0x7f49b00cc840 2024-04-24T22:39:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.386+0000 7f49a67fc640 1 -- 172.21.15.113:0/2490878044 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f49a800ad90 con 0x7f49b00cc840 2024-04-24T22:39:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.387+0000 7f49b808c640 1 -- 172.21.15.113:0/2490878044 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b0073b90 con 0x7f49b00cc840 2024-04-24T22:39:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.389+0000 7f49a67fc640 1 -- 172.21.15.113:0/2490878044 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f49a8021020 con 0x7f49b00cc840 2024-04-24T22:39:20.401 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:20.399+0000 7fee40928040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:39:20.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.460+0000 7f49b808c640 1 -- 172.21.15.113:0/2490878044 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "55580504-e9dc-47e0-841b-3a022c2b3328", "id": 5} v 0) -- 0x7f49b0153e50 con 0x7f49b00cc840 2024-04-24T22:39:20.466 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:20.463+0000 7faa8bc14040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T22:39:20.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.465+0000 7f49a67fc640 1 -- 172.21.15.113:0/2490878044 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "55580504-e9dc-47e0-841b-3a022c2b3328", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f49a800a640 con 0x7f49b00cc840 2024-04-24T22:39:20.467 INFO:teuthology.orchestra.run.smithi113.stdout:5 2024-04-24T22:39:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.466+0000 7f49b808c640 1 -- 172.21.15.113:0/2490878044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b00cc840 msgr2=0x7f49b0159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.466+0000 7f49b808c640 1 --2- 172.21.15.113:0/2490878044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b00cc840 0x7f49b0159150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f49a800c7b0 tx=0x7f49a800cc70 comp rx=0 tx=0).stop 2024-04-24T22:39:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.466+0000 7f49b808c640 1 -- 172.21.15.113:0/2490878044 shutdown_connections 2024-04-24T22:39:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.466+0000 7f49b808c640 1 --2- 172.21.15.113:0/2490878044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b00cc840 0x7f49b0159150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.466+0000 7f49b808c640 1 --2- 172.21.15.113:0/2490878044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b00cb6d0 0x7f49b01e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.466+0000 7f49b808c640 1 --2- 172.21.15.113:0/2490878044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0073380 0x7f49b00cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.466+0000 7f49b808c640 1 -- 172.21.15.113:0/2490878044 >> 172.21.15.113:0/2490878044 conn(0x7f49b006c940 msgr2=0x7f49b0153710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:20.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.466+0000 7f49b808c640 1 -- 172.21.15.113:0/2490878044 shutdown_connections 2024-04-24T22:39:20.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.466+0000 7f49b808c640 1 -- 172.21.15.113:0/2490878044 wait complete. 2024-04-24T22:39:20.478 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new a711f1d9-356f-43d9-b43e-849f4815402b 6 2024-04-24T22:39:20.519 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:20.516+0000 7faa8bc14040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T22:39:20.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.574+0000 7fe8e45a5640 1 Processor -- start 2024-04-24T22:39:20.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.574+0000 7fe8e45a5640 1 -- start start 2024-04-24T22:39:20.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e45a5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe8dc0c8360 0x7fe8dc0c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e45a5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8dc073310 con 0x7fe8dc152720 2024-04-24T22:39:20.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e45a5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8dc071bd0 con 0x7fe8dc06bce0 2024-04-24T22:39:20.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e45a5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe8dc06c150 con 0x7fe8dc0c8360 2024-04-24T22:39:20.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e2b42640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe8dc06bce0 0x7fe8dc162c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33246/0 (socket says 172.21.15.113:33246) 2024-04-24T22:39:20.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e2b42640 1 -- 172.21.15.113:0/1494001037 learned_addr learned my addr 172.21.15.113:0/1494001037 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T22:39:20.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2475114579 0 0) 0x7fe8dc071bd0 con 0x7fe8dc06bce0 2024-04-24T22:39:20.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8c8003520 con 0x7fe8dc06bce0 2024-04-24T22:39:20.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e1b40640 1 --2- 172.21.15.113:0/1494001037 >> v2:172.21.15.47:3300/0 conn(0x7fe8dc0c8360 0x7fe8dc0c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 552450852 0 0) 0x7fe8c8003520 con 0x7fe8dc06bce0 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 >> v2:172.21.15.47:3300/0 conn(0x7fe8dc0c8360 msgr2=0x7fe8dc0c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e133f640 1 --2- 172.21.15.113:0/1494001037 >> v2:172.21.15.47:3300/0 conn(0x7fe8dc0c8360 0x7fe8dc0c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 >> v1:172.21.15.47:6789/0 conn(0x7fe8dc152720 legacy=0x7fe8dc0c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc06c760 con 0x7fe8dc06bce0 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 567141988 0 0) 0x7fe8d40019f0 con 0x7fe8dc06bce0 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 >> v1:172.21.15.113:6789/0 conn(0x7fe8dc06bce0 legacy=0x7fe8dc162c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.575+0000 7fe8e133f640 1 --2- 172.21.15.113:0/1494001037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8c80039b0 0x7fe8c8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e133f640 1 --2- 172.21.15.113:0/1494001037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8c8004800 0x7fe8c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e133f640 1 --2- 172.21.15.113:0/1494001037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8c80051c0 0x7fe8c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc06c150 con 0x7fe8c80051c0 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c800c0f0 con 0x7fe8c80039b0 2024-04-24T22:39:20.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c800c2f0 con 0x7fe8c8004800 2024-04-24T22:39:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e2341640 1 --2- 172.21.15.113:0/1494001037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8c80039b0 0x7fe8c8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e1b40640 1 --2- 172.21.15.113:0/1494001037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8c8004800 0x7fe8c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e2341640 1 -- 172.21.15.113:0/1494001037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8c8004800 msgr2=0x7fe8c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e2341640 1 --2- 172.21.15.113:0/1494001037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8c8004800 0x7fe8c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e2341640 1 -- 172.21.15.113:0/1494001037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8c80051c0 msgr2=0x7fe8c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e2341640 1 --2- 172.21.15.113:0/1494001037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8c80051c0 0x7fe8c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e2341640 1 -- 172.21.15.113:0/1494001037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c8003520 con 0x7fe8c80039b0 2024-04-24T22:39:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.576+0000 7fe8e2b42640 1 -- 172.21.15.113:0/1494001037 reap_dead start 2024-04-24T22:39:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e2341640 1 --2- 172.21.15.113:0/1494001037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8c80039b0 0x7fe8c8003db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8003d80 tx=0x7fe8c8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4018cd09381827f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8037070 con 0x7fe8c80039b0 2024-04-24T22:39:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d802a9d0 con 0x7fe8c80039b0 2024-04-24T22:39:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e133f640 1 -- 172.21.15.113:0/1494001037 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8003980 con 0x7fe8c80039b0 2024-04-24T22:39:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e45a5640 1 -- 172.21.15.113:0/1494001037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8c80039b0 msgr2=0x7fe8c8003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e45a5640 1 --2- 172.21.15.113:0/1494001037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8c80039b0 0x7fe8c8003db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8003d80 tx=0x7fe8c8005830 comp rx=0 tx=0).stop 2024-04-24T22:39:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e45a5640 1 -- 172.21.15.113:0/1494001037 shutdown_connections 2024-04-24T22:39:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e45a5640 1 --2- 172.21.15.113:0/1494001037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8c80039b0 0x7fe8c8003db0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e45a5640 1 -- 172.21.15.113:0/1494001037 >> 172.21.15.113:0/1494001037 conn(0x7fe8dc06f520 msgr2=0x7fe8dc06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e45a5640 1 -- 172.21.15.113:0/1494001037 shutdown_connections 2024-04-24T22:39:20.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.577+0000 7fe8e45a5640 1 -- 172.21.15.113:0/1494001037 wait complete. 2024-04-24T22:39:20.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e45a5640 1 Processor -- start 2024-04-24T22:39:20.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e45a5640 1 -- start start 2024-04-24T22:39:20.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e45a5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8c80039b0 0x7fe8dc1ee490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e45a5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8dc154300 0x7fe8dc149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e45a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8dc14d1c0 0x7fe8dc152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e45a5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc06c510 con 0x7fe8c80039b0 2024-04-24T22:39:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e45a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc071980 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e45a5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe8dc073860 con 0x7fe8dc154300 2024-04-24T22:39:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e2b42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8dc14d1c0 0x7fe8dc152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e2b42640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8dc14d1c0 0x7fe8dc152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53856/0 (socket says 172.21.15.113:53856) 2024-04-24T22:39:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e1b40640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8dc154300 0x7fe8dc149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.579+0000 7fe8e2b42640 1 -- 172.21.15.113:0/2933560455 learned_addr learned my addr 172.21.15.113:0/2933560455 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8e2b42640 1 -- 172.21.15.113:0/2933560455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8dc154300 msgr2=0x7fe8dc149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8e2b42640 1 --2- 172.21.15.113:0/2933560455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8dc154300 0x7fe8dc149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8e2b42640 1 -- 172.21.15.113:0/2933560455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8c80039b0 msgr2=0x7fe8dc1ee490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8e2b42640 1 --2- 172.21.15.113:0/2933560455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8c80039b0 0x7fe8dc1ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8e2b42640 1 -- 172.21.15.113:0/2933560455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc154a30 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8e2b42640 1 --2- 172.21.15.113:0/2933560455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8dc14d1c0 0x7fe8dc152bc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc06eb00 tx=0x7fe8d400ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8d2ffd640 1 -- 172.21.15.113:0/2933560455 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400a5b0 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8d2ffd640 1 -- 172.21.15.113:0/2933560455 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d400a750 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8d2ffd640 1 -- 172.21.15.113:0/2933560455 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400ab20 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8e45a5640 1 -- 172.21.15.113:0/2933560455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8dc154cf0 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8e45a5640 1 -- 172.21.15.113:0/2933560455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8dc1552b0 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8d2ffd640 1 -- 172.21.15.113:0/2933560455 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8d4011070 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.580+0000 7fe8d2ffd640 1 -- 172.21.15.113:0/2933560455 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fe8d4013d70 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.581+0000 7fe8e45a5640 1 -- 172.21.15.113:0/2933560455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8dc073b90 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.583+0000 7fe8d2ffd640 1 -- 172.21.15.113:0/2933560455 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe8d4019060 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.656+0000 7fe8e45a5640 1 -- 172.21.15.113:0/2933560455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a711f1d9-356f-43d9-b43e-849f4815402b", "id": 6} v 0) -- 0x7fe8dc0c6bb0 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.661+0000 7fe8d2ffd640 1 -- 172.21.15.113:0/2933560455 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a711f1d9-356f-43d9-b43e-849f4815402b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fe8d4013410 con 0x7fe8dc14d1c0 2024-04-24T22:39:20.663 INFO:teuthology.orchestra.run.smithi113.stdout:6 2024-04-24T22:39:20.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.662+0000 7fe8e45a5640 1 -- 172.21.15.113:0/2933560455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8dc14d1c0 msgr2=0x7fe8dc152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.662+0000 7fe8e45a5640 1 --2- 172.21.15.113:0/2933560455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8dc14d1c0 0x7fe8dc152bc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc06eb00 tx=0x7fe8d400ccd0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.662+0000 7fe8e45a5640 1 -- 172.21.15.113:0/2933560455 shutdown_connections 2024-04-24T22:39:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.662+0000 7fe8e45a5640 1 --2- 172.21.15.113:0/2933560455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8dc14d1c0 0x7fe8dc152bc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.662+0000 7fe8e45a5640 1 --2- 172.21.15.113:0/2933560455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8dc154300 0x7fe8dc149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.662+0000 7fe8e45a5640 1 --2- 172.21.15.113:0/2933560455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8c80039b0 0x7fe8dc1ee490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.662+0000 7fe8e45a5640 1 -- 172.21.15.113:0/2933560455 >> 172.21.15.113:0/2933560455 conn(0x7fe8dc06f520 msgr2=0x7fe8dc150740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.662+0000 7fe8e45a5640 1 -- 172.21.15.113:0/2933560455 shutdown_connections 2024-04-24T22:39:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.662+0000 7fe8e45a5640 1 -- 172.21.15.113:0/2933560455 wait complete. 2024-04-24T22:39:20.674 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new cc5ddb38-cf05-42cd-b3d8-41f011a14345 7 2024-04-24T22:39:20.720 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:20.717+0000 7faa8bc14040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T22:39:20.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.768+0000 7fd4dba3f640 1 Processor -- start 2024-04-24T22:39:20.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.769+0000 7fd4dba3f640 1 -- start start 2024-04-24T22:39:20.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.769+0000 7fd4dba3f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d40c9300 0x7fd4d40c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.769+0000 7fd4dba3f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd4d415b180 0x7fd4d40ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.769+0000 7fd4dba3f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d415a6c0 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.769+0000 7fd4dba3f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d415b610 con 0x7fd4d415b180 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.769+0000 7fd4dba3f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d415acf0 con 0x7fd4d40c9300 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.769+0000 7fd4d97db640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d40c9300 0x7fd4d40c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.769+0000 7fd4d8fda640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd4d40c9ed0 0x7fd4d415dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33256/0 (socket says 172.21.15.113:33256) 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.770+0000 7fd4d8fda640 1 -- 172.21.15.113:0/288933632 learned_addr learned my addr 172.21.15.113:0/288933632 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.770+0000 7fd4cbfff640 1 -- 172.21.15.113:0/288933632 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1155411448 0 0) 0x7fd4d415a6c0 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.770+0000 7fd4cbfff640 1 -- 172.21.15.113:0/288933632 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4c0003520 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.770+0000 7fd4d97db640 1 -- 172.21.15.113:0/288933632 >> v2:172.21.15.47:3300/0 conn(0x7fd4d415b180 msgr2=0x7fd4d40ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.770+0000 7fd4d97db640 1 --2- 172.21.15.113:0/288933632 >> v2:172.21.15.47:3300/0 conn(0x7fd4d415b180 0x7fd4d40ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.770+0000 7fd4d97db640 1 -- 172.21.15.113:0/288933632 >> v1:172.21.15.113:6789/0 conn(0x7fd4d40c9ed0 legacy=0x7fd4d415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.770+0000 7fd4d97db640 1 -- 172.21.15.113:0/288933632 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d415e3e0 con 0x7fd4d40c9300 2024-04-24T22:39:20.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4d97db640 1 --2- 172.21.15.113:0/288933632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d40c9300 0x7fd4d40c8590 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40097a0 tx=0x7fd4c402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30a7c832ebf2acf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:20.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4cbfff640 1 -- 172.21.15.113:0/288933632 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4037070 con 0x7fd4d40c9300 2024-04-24T22:39:20.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4cbfff640 1 -- 172.21.15.113:0/288933632 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c4032440 con 0x7fd4d40c9300 2024-04-24T22:39:20.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4cbfff640 1 -- 172.21.15.113:0/288933632 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4032810 con 0x7fd4d40c9300 2024-04-24T22:39:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4dba3f640 1 -- 172.21.15.113:0/288933632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d40c9300 msgr2=0x7fd4d40c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4dba3f640 1 --2- 172.21.15.113:0/288933632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d40c9300 0x7fd4d40c8590 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40097a0 tx=0x7fd4c402a180 comp rx=0 tx=0).stop 2024-04-24T22:39:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4dba3f640 1 -- 172.21.15.113:0/288933632 shutdown_connections 2024-04-24T22:39:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4dba3f640 1 --2- 172.21.15.113:0/288933632 >> v2:172.21.15.47:3300/0 conn(0x7fd4d415b180 0x7fd4d40ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4dba3f640 1 --2- 172.21.15.113:0/288933632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d40c9300 0x7fd4d40c8590 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4dba3f640 1 -- 172.21.15.113:0/288933632 >> 172.21.15.113:0/288933632 conn(0x7fd4d4073420 msgr2=0x7fd4d4070730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4dba3f640 1 -- 172.21.15.113:0/288933632 shutdown_connections 2024-04-24T22:39:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.771+0000 7fd4dba3f640 1 -- 172.21.15.113:0/288933632 wait complete. 2024-04-24T22:39:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4dba3f640 1 Processor -- start 2024-04-24T22:39:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4dba3f640 1 -- start start 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4dba3f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d40c9300 0x7fd4d41f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4dba3f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d40c9ed0 0x7fd4d41e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4dba3f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d415b180 0x7fd4d4157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4dba3f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d415a980 con 0x7fd4d40c9300 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4dba3f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d415e9a0 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4dba3f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d40c96e0 con 0x7fd4d415b180 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4d9fdc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d415b180 0x7fd4d4157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4d8fda640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d40c9ed0 0x7fd4d41e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4d9fdc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d415b180 0x7fd4d4157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:44508/0 (socket says 172.21.15.113:44508) 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4d8fda640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d40c9ed0 0x7fd4d41e4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53860/0 (socket says 172.21.15.113:53860) 2024-04-24T22:39:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.773+0000 7fd4d9fdc640 1 -- 172.21.15.113:0/3412631812 learned_addr learned my addr 172.21.15.113:0/3412631812 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4d8fda640 1 -- 172.21.15.113:0/3412631812 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d415b180 msgr2=0x7fd4d4157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4d8fda640 1 --2- 172.21.15.113:0/3412631812 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d415b180 0x7fd4d4157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4d8fda640 1 -- 172.21.15.113:0/3412631812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d40c9300 msgr2=0x7fd4d41f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4d8fda640 1 --2- 172.21.15.113:0/3412631812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d40c9300 0x7fd4d41f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4d8fda640 1 -- 172.21.15.113:0/3412631812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d4157fd0 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4d9fdc640 1 --2- 172.21.15.113:0/3412631812 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d415b180 0x7fd4d4157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4d8fda640 1 --2- 172.21.15.113:0/3412631812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d40c9ed0 0x7fd4d41e4100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000c400 tx=0x7fd4d000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4c9ffb640 1 -- 172.21.15.113:0/3412631812 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0017070 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4c9ffb640 1 -- 172.21.15.113:0/3412631812 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d0005ce0 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4dba3f640 1 -- 172.21.15.113:0/3412631812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d4156710 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4c9ffb640 1 -- 172.21.15.113:0/3412631812 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00072f0 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4c9ffb640 1 -- 172.21.15.113:0/3412631812 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4d001a070 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4dba3f640 1 -- 172.21.15.113:0/3412631812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d4156cd0 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.774+0000 7fd4c9ffb640 1 -- 172.21.15.113:0/3412631812 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fd4d0012cd0 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.775+0000 7fd4dba3f640 1 -- 172.21.15.113:0/3412631812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d4073b90 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.777+0000 7fd4c9ffb640 1 -- 172.21.15.113:0/3412631812 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd4d0022080 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.785 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:20.782+0000 7faa8bc14040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T22:39:20.804 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:20.802+0000 7fee40928040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:39:20.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.847+0000 7fd4dba3f640 1 -- 172.21.15.113:0/3412631812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cc5ddb38-cf05-42cd-b3d8-41f011a14345", "id": 7} v 0) -- 0x7fd4d40cb410 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.849 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:20.847+0000 7faa8bc14040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T22:39:20.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.852+0000 7fd4c9ffb640 1 -- 172.21.15.113:0/3412631812 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cc5ddb38-cf05-42cd-b3d8-41f011a14345", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fd4d0007560 con 0x7fd4d40c9ed0 2024-04-24T22:39:20.854 INFO:teuthology.orchestra.run.smithi113.stdout:7 2024-04-24T22:39:20.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.853+0000 7fd4dba3f640 1 -- 172.21.15.113:0/3412631812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d40c9ed0 msgr2=0x7fd4d41e4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:20.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.853+0000 7fd4dba3f640 1 --2- 172.21.15.113:0/3412631812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d40c9ed0 0x7fd4d41e4100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000c400 tx=0x7fd4d000c8c0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.853+0000 7fd4dba3f640 1 -- 172.21.15.113:0/3412631812 shutdown_connections 2024-04-24T22:39:20.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.853+0000 7fd4dba3f640 1 --2- 172.21.15.113:0/3412631812 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d415b180 0x7fd4d4157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.853+0000 7fd4dba3f640 1 --2- 172.21.15.113:0/3412631812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d40c9ed0 0x7fd4d41e4100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.853+0000 7fd4dba3f640 1 --2- 172.21.15.113:0/3412631812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4d40c9300 0x7fd4d41f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:20.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.853+0000 7fd4dba3f640 1 -- 172.21.15.113:0/3412631812 >> 172.21.15.113:0/3412631812 conn(0x7fd4d4073420 msgr2=0x7fd4d40ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:20.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.853+0000 7fd4dba3f640 1 -- 172.21.15.113:0/3412631812 shutdown_connections 2024-04-24T22:39:20.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-24T22:39:20.853+0000 7fd4dba3f640 1 -- 172.21.15.113:0/3412631812 wait complete. 2024-04-24T22:39:20.865 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-24T22:39:20.866 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-24T22:39:20.872 INFO:tasks.ceph.osd.0:Started 2024-04-24T22:39:20.872 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-24T22:39:20.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-24T22:39:20.877 INFO:tasks.ceph.osd.1:Started 2024-04-24T22:39:20.877 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-24T22:39:20.878 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-24T22:39:20.883 INFO:tasks.ceph.osd.2:Started 2024-04-24T22:39:20.883 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-24T22:39:20.883 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-24T22:39:20.888 INFO:tasks.ceph.osd.3:Started 2024-04-24T22:39:20.888 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-24T22:39:20.889 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-24T22:39:20.890 INFO:tasks.ceph.osd.4:Started 2024-04-24T22:39:20.891 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-24T22:39:20.891 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-24T22:39:20.892 INFO:tasks.ceph.osd.5:Started 2024-04-24T22:39:20.893 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T22:39:20.893 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-24T22:39:20.894 INFO:tasks.ceph.osd.6:Started 2024-04-24T22:39:20.895 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-24T22:39:20.895 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-24T22:39:20.897 INFO:tasks.ceph.osd.7:Started 2024-04-24T22:39:20.897 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:21.046 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:21.043+0000 7faa8bc14040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T22:39:21.082 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:21.080+0000 7fee40928040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:39:21.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.124+0000 7f31e8a00640 1 Processor -- start 2024-04-24T22:39:21.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.124+0000 7f31e8a00640 1 -- start start 2024-04-24T22:39:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.126+0000 7f31e8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e40c2e80 0x7f31e4072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.126+0000 7f31e8a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e40c8c90 con 0x7f31e40c7fc0 2024-04-24T22:39:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.126+0000 7f31e8a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e4152c10 con 0x7f31e4152830 2024-04-24T22:39:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.126+0000 7f31e8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f31e40c8760 con 0x7f31e40c2e80 2024-04-24T22:39:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.126+0000 7f31e1e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f31e4152830 0x7f31e406d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:55886/0 (socket says 172.21.15.47:55886) 2024-04-24T22:39:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.126+0000 7f31e1e00640 1 -- 172.21.15.47:0/3594871014 learned_addr learned my addr 172.21.15.47:0/3594871014 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.126+0000 7f31e1400640 1 --2- 172.21.15.47:0/3594871014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e40c2e80 0x7f31e4072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.126+0000 7f31e0a00640 1 -- 172.21.15.47:0/3594871014 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52962542 0 0) 0x7f31e4152c10 con 0x7f31e4152830 2024-04-24T22:39:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.126+0000 7f31e0a00640 1 -- 172.21.15.47:0/3594871014 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31c8003520 con 0x7f31e4152830 2024-04-24T22:39:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.127+0000 7f31e1400640 1 -- 172.21.15.47:0/3594871014 >> v1:172.21.15.113:6789/0 conn(0x7f31e4152830 legacy=0x7f31e406d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.127+0000 7f31e1400640 1 -- 172.21.15.47:0/3594871014 >> v1:172.21.15.47:6789/0 conn(0x7f31e40c7fc0 legacy=0x7f31e4162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.127+0000 7f31e1400640 1 -- 172.21.15.47:0/3594871014 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e40c89b0 con 0x7f31e40c2e80 2024-04-24T22:39:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.128+0000 7f31e1400640 1 --2- 172.21.15.47:0/3594871014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e40c2e80 0x7f31e4072e90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009750 tx=0x7f31d002a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7adde17c8e87a9c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.128+0000 7f31e0a00640 1 -- 172.21.15.47:0/3594871014 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0037070 con 0x7f31e40c2e80 2024-04-24T22:39:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.128+0000 7f31e0a00640 1 -- 172.21.15.47:0/3594871014 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d002aad0 con 0x7f31e40c2e80 2024-04-24T22:39:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.128+0000 7f31e0a00640 1 -- 172.21.15.47:0/3594871014 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4001670 con 0x7f31e40c2e80 2024-04-24T22:39:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.128+0000 7f31e8a00640 1 -- 172.21.15.47:0/3594871014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e40c2e80 msgr2=0x7f31e4072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.128+0000 7f31e8a00640 1 --2- 172.21.15.47:0/3594871014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e40c2e80 0x7f31e4072e90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009750 tx=0x7f31d002a040 comp rx=0 tx=0).stop 2024-04-24T22:39:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.129+0000 7f31e8a00640 1 -- 172.21.15.47:0/3594871014 shutdown_connections 2024-04-24T22:39:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.129+0000 7f31e8a00640 1 --2- 172.21.15.47:0/3594871014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e40c2e80 0x7f31e4072e90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.129+0000 7f31e8a00640 1 -- 172.21.15.47:0/3594871014 >> 172.21.15.47:0/3594871014 conn(0x7f31e406e730 msgr2=0x7f31e406eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.129+0000 7f31e8a00640 1 -- 172.21.15.47:0/3594871014 shutdown_connections 2024-04-24T22:39:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.129+0000 7f31e8a00640 1 -- 172.21.15.47:0/3594871014 wait complete. 2024-04-24T22:39:21.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.131+0000 7f31e8a00640 1 Processor -- start 2024-04-24T22:39:21.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.131+0000 7f31e8a00640 1 -- start start 2024-04-24T22:39:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e40c2e80 0x7f31e41e7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31e40c7fc0 0x7f31e41f0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e4152830 0x7f31e41f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e8a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40ca480 con 0x7f31e40c7fc0 2024-04-24T22:39:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f31e406e300 con 0x7f31e40c2e80 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f31e40c4ca0 con 0x7f31e4152830 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e40c2e80 0x7f31e41e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e40c2e80 0x7f31e41e7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:46550/0 (socket says 172.21.15.47:46550) 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e1e00640 1 -- 172.21.15.47:0/3780933831 learned_addr learned my addr 172.21.15.47:0/3780933831 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e2800640 1 --2- 172.21.15.47:0/3780933831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e4152830 0x7f31e41f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e1400640 1 --2- 172.21.15.47:0/3780933831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31e40c7fc0 0x7f31e41f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e1e00640 1 -- 172.21.15.47:0/3780933831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e4152830 msgr2=0x7f31e41f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e1e00640 1 --2- 172.21.15.47:0/3780933831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e4152830 0x7f31e41f0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e1e00640 1 -- 172.21.15.47:0/3780933831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31e40c7fc0 msgr2=0x7f31e41f0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e1e00640 1 --2- 172.21.15.47:0/3780933831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31e40c7fc0 0x7f31e41f0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e1e00640 1 -- 172.21.15.47:0/3780933831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e41564c0 con 0x7f31e40c2e80 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.132+0000 7f31e1400640 1 --2- 172.21.15.47:0/3780933831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31e40c7fc0 0x7f31e41f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.133+0000 7f31e1e00640 1 --2- 172.21.15.47:0/3780933831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e40c2e80 0x7f31e41e7270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f31d400c7b0 tx=0x7f31d400cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.133+0000 7f31da000640 1 -- 172.21.15.47:0/3780933831 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4018070 con 0x7f31e40c2e80 2024-04-24T22:39:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.133+0000 7f31da000640 1 -- 172.21.15.47:0/3780933831 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d40074e0 con 0x7f31e40c2e80 2024-04-24T22:39:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.133+0000 7f31e8a00640 1 -- 172.21.15.47:0/3780933831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e41567a0 con 0x7f31e40c2e80 2024-04-24T22:39:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.133+0000 7f31da000640 1 -- 172.21.15.47:0/3780933831 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d400ce60 con 0x7f31e40c2e80 2024-04-24T22:39:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.133+0000 7f31e8a00640 1 -- 172.21.15.47:0/3780933831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e4156d60 con 0x7f31e40c2e80 2024-04-24T22:39:21.137 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:21.134+0000 7fee40928040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:39:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.133+0000 7f31da000640 1 -- 172.21.15.47:0/3780933831 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f31d401b070 con 0x7f31e40c2e80 2024-04-24T22:39:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.133+0000 7f31da000640 1 -- 172.21.15.47:0/3780933831 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f31d400adf0 con 0x7f31e40c2e80 2024-04-24T22:39:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.134+0000 7f31e8a00640 1 -- 172.21.15.47:0/3780933831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e4073b90 con 0x7f31e40c2e80 2024-04-24T22:39:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.136+0000 7f31da000640 1 -- 172.21.15.47:0/3780933831 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f31d4022020 con 0x7f31e40c2e80 2024-04-24T22:39:21.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.203+0000 7f31e8a00640 1 -- 172.21.15.47:0/3780933831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31e40c9420 con 0x7f31e40c2e80 2024-04-24T22:39:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.204+0000 7f31da000640 1 -- 172.21.15.47:0/3780933831 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f31d400a7b0 con 0x7f31e40c2e80 2024-04-24T22:39:21.206 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:21.206 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:20.849444+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.205+0000 7f31e8a00640 1 -- 172.21.15.47:0/3780933831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e40c2e80 msgr2=0x7f31e41e7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.205+0000 7f31e8a00640 1 --2- 172.21.15.47:0/3780933831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e40c2e80 0x7f31e41e7270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f31d400c7b0 tx=0x7f31d400cc70 comp rx=0 tx=0).stop 2024-04-24T22:39:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.205+0000 7f31e8a00640 1 -- 172.21.15.47:0/3780933831 shutdown_connections 2024-04-24T22:39:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.205+0000 7f31e8a00640 1 --2- 172.21.15.47:0/3780933831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31e4152830 0x7f31e41f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.205+0000 7f31e8a00640 1 --2- 172.21.15.47:0/3780933831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31e40c7fc0 0x7f31e41f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.205+0000 7f31e8a00640 1 --2- 172.21.15.47:0/3780933831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31e40c2e80 0x7f31e41e7270 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.205+0000 7f31e8a00640 1 -- 172.21.15.47:0/3780933831 >> 172.21.15.47:0/3780933831 conn(0x7f31e406e730 msgr2=0x7f31e406eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.205+0000 7f31e8a00640 1 -- 172.21.15.47:0/3780933831 shutdown_connections 2024-04-24T22:39:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:21.205+0000 7f31e8a00640 1 -- 172.21.15.47:0/3780933831 wait complete. 2024-04-24T22:39:21.223 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-24T22:39:21.223 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-24T22:39:21.253 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:21.251+0000 7fee40928040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:39:21.490 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-24T22:39:21.488+0000 7f9ca880f640 -1 Falling back to public interface 2024-04-24T22:39:21.495 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-04-24T22:39:21.493+0000 7f3d70c0d640 -1 Falling back to public interface 2024-04-24T22:39:21.507 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-04-24T22:39:21.505+0000 7f37a2a1a640 -1 Falling back to public interface 2024-04-24T22:39:21.507 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-04-24T22:39:21.505+0000 7f599aaea640 -1 Falling back to public interface 2024-04-24T22:39:21.534 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:21.531+0000 7faa8bc14040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T22:39:21.534 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:21.531+0000 7fee40928040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:39:21.577 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-24T22:39:21.575+0000 7fc2d9c1e640 -1 Falling back to public interface 2024-04-24T22:39:21.581 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:21.580+0000 7fee40928040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:39:21.597 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-24T22:39:21.595+0000 7f23e20f0640 -1 Falling back to public interface 2024-04-24T22:39:21.605 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-24T22:39:21.603+0000 7fb7a3bc0640 -1 Falling back to public interface 2024-04-24T22:39:21.623 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-24T22:39:21.620+0000 7f702552c640 -1 Falling back to public interface 2024-04-24T22:39:21.736 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:21.734+0000 7fee40928040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:39:21.847 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:21.844+0000 7faa8bc14040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T22:39:21.873 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:21.871+0000 7fee40928040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:39:21.910 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:21.908+0000 7faa8bc14040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T22:39:21.922 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:21.920+0000 7fee40928040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:39:21.990 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:21.988+0000 7fee40928040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:39:22.029 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:22.034 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:22.033+0000 7fee40928040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:39:22.049 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:22.046+0000 7faa8bc14040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T22:39:22.190 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.187+0000 7f0e2de00640 1 Processor -- start 2024-04-24T22:39:22.190 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.187+0000 7f0e2de00640 1 -- start start 2024-04-24T22:39:22.192 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.189+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815b1d0 0x7f0e2806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:22.192 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.189+0000 7f0e2de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e2806c070 0x7f0e2806c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:22.192 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.189+0000 7f0e2de00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2806b220 con 0x7f0e28154540 2024-04-24T22:39:22.192 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.189+0000 7f0e2de00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2806b500 con 0x7f0e2806c070 2024-04-24T22:39:22.192 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.189+0000 7f0e2de00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2815a6f0 con 0x7f0e2815b1d0 2024-04-24T22:39:22.193 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.189+0000 7f0e2d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e2806c070 0x7f0e2806c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:22.193 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.189+0000 7f0e2ca00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0e28154540 0x7f0e2815dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:55896/0 (socket says 172.21.15.47:55896) 2024-04-24T22:39:22.193 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.189+0000 7f0e2ca00640 1 -- 172.21.15.47:0/2853111072 learned_addr learned my addr 172.21.15.47:0/2853111072 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:22.193 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.189+0000 7f0e27e00640 1 --2- 172.21.15.47:0/2853111072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815b1d0 0x7f0e2806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:22.193 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.190+0000 7f0e2d400640 1 -- 172.21.15.47:0/2853111072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815b1d0 msgr2=0x7f0e2806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:22.193 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.190+0000 7f0e2d400640 1 --2- 172.21.15.47:0/2853111072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815b1d0 0x7f0e2806bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:22.193 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.190+0000 7f0e2d400640 1 -- 172.21.15.47:0/2853111072 >> v1:172.21.15.113:6789/0 conn(0x7f0e28154540 legacy=0x7f0e2815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:22.193 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.190+0000 7f0e2d400640 1 -- 172.21.15.47:0/2853111072 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2815a9d0 con 0x7f0e2806c070 2024-04-24T22:39:22.193 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.190+0000 7f0e27400640 1 -- 172.21.15.47:0/2853111072 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982249759 0 0) 0x7f0e2806b220 con 0x7f0e28154540 2024-04-24T22:39:22.195 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.191+0000 7f0e2d400640 1 --2- 172.21.15.47:0/2853111072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 0x7f0e2806c640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009d90 tx=0x7f0e2002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a2a2bef84d8261 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:22.195 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.191+0000 7f0e27400640 1 -- 172.21.15.47:0/2853111072 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20007740 con 0x7f0e2806c070 2024-04-24T22:39:22.195 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.191+0000 7f0e27400640 1 -- 172.21.15.47:0/2853111072 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e200078e0 con 0x7f0e2806c070 2024-04-24T22:39:22.196 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.191+0000 7f0e27400640 1 -- 172.21.15.47:0/2853111072 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20007cb0 con 0x7f0e2806c070 2024-04-24T22:39:22.196 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.192+0000 7f0e2de00640 1 -- 172.21.15.47:0/2853111072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 msgr2=0x7f0e2806c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:22.196 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.192+0000 7f0e2de00640 1 --2- 172.21.15.47:0/2853111072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 0x7f0e2806c640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009d90 tx=0x7f0e2002a660 comp rx=0 tx=0).stop 2024-04-24T22:39:22.196 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.192+0000 7f0e2de00640 1 -- 172.21.15.47:0/2853111072 shutdown_connections 2024-04-24T22:39:22.196 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.192+0000 7f0e2de00640 1 --2- 172.21.15.47:0/2853111072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 0x7f0e2806c640 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:22.196 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.192+0000 7f0e2de00640 1 --2- 172.21.15.47:0/2853111072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815b1d0 0x7f0e2806bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:22.196 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.192+0000 7f0e2de00640 1 -- 172.21.15.47:0/2853111072 >> 172.21.15.47:0/2853111072 conn(0x7f0e28072c00 msgr2=0x7f0e28073010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:22.196 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.192+0000 7f0e2de00640 1 -- 172.21.15.47:0/2853111072 shutdown_connections 2024-04-24T22:39:22.196 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.192+0000 7f0e2de00640 1 -- 172.21.15.47:0/2853111072 wait complete. 2024-04-24T22:39:22.199 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.193+0000 7f0e2de00640 1 Processor -- start 2024-04-24T22:39:22.199 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.194+0000 7f0e2de00640 1 -- start start 2024-04-24T22:39:22.199 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.194+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 0x7f0e281f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:22.199 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e28154540 0x7f0e28159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:22.199 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815b1d0 0x7f0e281564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:22.199 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2de00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0e280cc680 con 0x7f0e28154540 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2de00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e280cc500 con 0x7f0e2806c070 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2de00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2806cb80 con 0x7f0e2815b1d0 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 0x7f0e281f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 0x7f0e281f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:46572/0 (socket says 172.21.15.47:46572) 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2ca00640 1 -- 172.21.15.47:0/3184067166 learned_addr learned my addr 172.21.15.47:0/3184067166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2d400640 1 --2- 172.21.15.47:0/3184067166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815b1d0 0x7f0e281564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e27e00640 1 --2- 172.21.15.47:0/3184067166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e28154540 0x7f0e28159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2ca00640 1 -- 172.21.15.47:0/3184067166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815b1d0 msgr2=0x7f0e281564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2ca00640 1 --2- 172.21.15.47:0/3184067166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815b1d0 0x7f0e281564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2ca00640 1 -- 172.21.15.47:0/3184067166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e28154540 msgr2=0x7f0e28159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2ca00640 1 --2- 172.21.15.47:0/3184067166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e28154540 0x7f0e28159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.195+0000 7f0e2ca00640 1 -- 172.21.15.47:0/3184067166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e28153a40 con 0x7f0e2806c070 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.196+0000 7f0e2ca00640 1 --2- 172.21.15.47:0/3184067166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 0x7f0e281f4180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00c4d0 tx=0x7f0e1c00c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.196+0000 7f0e24c00640 1 -- 172.21.15.47:0/3184067166 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c018070 con 0x7f0e2806c070 2024-04-24T22:39:22.200 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.196+0000 7f0e24c00640 1 -- 172.21.15.47:0/3184067166 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0074e0 con 0x7f0e2806c070 2024-04-24T22:39:22.201 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.196+0000 7f0e24c00640 1 -- 172.21.15.47:0/3184067166 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c004a20 con 0x7f0e2806c070 2024-04-24T22:39:22.201 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.196+0000 7f0e2de00640 1 -- 172.21.15.47:0/3184067166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e281f10d0 con 0x7f0e2806c070 2024-04-24T22:39:22.201 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.196+0000 7f0e2de00640 1 -- 172.21.15.47:0/3184067166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e281f0310 con 0x7f0e2806c070 2024-04-24T22:39:22.201 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.196+0000 7f0e24c00640 1 -- 172.21.15.47:0/3184067166 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e1c004bc0 con 0x7f0e2806c070 2024-04-24T22:39:22.201 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.196+0000 7f0e24c00640 1 -- 172.21.15.47:0/3184067166 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0e1c01b360 con 0x7f0e2806c070 2024-04-24T22:39:22.201 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.197+0000 7f0e2de00640 1 -- 172.21.15.47:0/3184067166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e280758a0 con 0x7f0e2806c070 2024-04-24T22:39:22.202 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.200+0000 7f0e24c00640 1 -- 172.21.15.47:0/3184067166 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0e1c021020 con 0x7f0e2806c070 2024-04-24T22:39:22.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.267+0000 7f0e2de00640 1 -- 172.21.15.47:0/3184067166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e2806b910 con 0x7f0e2806c070 2024-04-24T22:39:22.270 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.267+0000 7f0e24c00640 1 -- 172.21.15.47:0/3184067166 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0e1c012400 con 0x7f0e2806c070 2024-04-24T22:39:22.271 INFO:teuthology.misc.health.smithi047.stdout: 2024-04-24T22:39:22.271 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":10,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:20.849444+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:22.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.269+0000 7f0e2de00640 1 -- 172.21.15.47:0/3184067166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 msgr2=0x7f0e281f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:22.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.269+0000 7f0e2de00640 1 --2- 172.21.15.47:0/3184067166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 0x7f0e281f4180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00c4d0 tx=0x7f0e1c00c990 comp rx=0 tx=0).stop 2024-04-24T22:39:22.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.269+0000 7f0e2de00640 1 -- 172.21.15.47:0/3184067166 shutdown_connections 2024-04-24T22:39:22.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.269+0000 7f0e2de00640 1 --2- 172.21.15.47:0/3184067166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815b1d0 0x7f0e281564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:22.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.269+0000 7f0e2de00640 1 --2- 172.21.15.47:0/3184067166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e28154540 0x7f0e28159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:22.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.269+0000 7f0e2de00640 1 --2- 172.21.15.47:0/3184067166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806c070 0x7f0e281f4180 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:22.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.269+0000 7f0e2de00640 1 -- 172.21.15.47:0/3184067166 >> 172.21.15.47:0/3184067166 conn(0x7f0e28072c00 msgr2=0x7f0e280b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:22.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.269+0000 7f0e2de00640 1 -- 172.21.15.47:0/3184067166 shutdown_connections 2024-04-24T22:39:22.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:22.269+0000 7f0e2de00640 1 -- 172.21.15.47:0/3184067166 wait complete. 2024-04-24T22:39:22.284 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T22:39:22.409 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:22.407+0000 7faa8bc14040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T22:39:22.469 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:22.466+0000 7faa8bc14040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T22:39:22.510 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:22.509+0000 7fee40928040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:39:22.628 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:22.626+0000 7fee40928040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:39:22.664 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:22.661+0000 7faa8bc14040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T22:39:22.842 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:22.840+0000 7faa8bc14040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T22:39:22.903 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:22.900+0000 7faa8bc14040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T22:39:22.988 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:22.985+0000 7faa8bc14040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T22:39:23.044 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:23.042+0000 7faa8bc14040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T22:39:23.609 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:23.606+0000 7faa8bc14040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T22:39:23.651 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:23.650+0000 7fede1450640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T22:39:23.652 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-04-24T22:39:23.650+0000 7fede1450640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T22:39:23.747 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-24T22:39:23.744+0000 7faa8bc14040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T22:39:24.981 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-24T22:39:24.979+0000 7f9ca880f640 -1 osd.5 0 log_to_monitors true 2024-04-24T22:39:24.981 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-04-24T22:39:24.979+0000 7f3d70c0d640 -1 osd.6 0 log_to_monitors true 2024-04-24T22:39:25.025 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-04-24T22:39:25.023+0000 7f37a2a1a640 -1 osd.7 0 log_to_monitors true 2024-04-24T22:39:25.031 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-04-24T22:39:25.029+0000 7f599aaea640 -1 osd.4 0 log_to_monitors true 2024-04-24T22:39:25.207 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-24T22:39:25.204+0000 7fc2d9c1e640 -1 osd.1 0 log_to_monitors true 2024-04-24T22:39:25.233 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-24T22:39:25.231+0000 7fb7a3bc0640 -1 osd.2 0 log_to_monitors true 2024-04-24T22:39:25.510 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-24T22:39:25.507+0000 7f23e20f0640 -1 osd.0 0 log_to_monitors true 2024-04-24T22:39:25.510 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-24T22:39:25.508+0000 7f702552c640 -1 osd.3 0 log_to_monitors true 2024-04-24T22:39:28.134 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-24T22:39:28.132+0000 7f9ca3901640 -1 osd.5 0 waiting for initial osdmap 2024-04-24T22:39:28.142 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-04-24T22:39:28.141+0000 7f379f3fc640 -1 osd.7 0 waiting for initial osdmap 2024-04-24T22:39:28.149 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-24T22:39:28.147+0000 7f9c9bf33640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:39:28.149 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-04-24T22:39:28.148+0000 7f3d6be15640 -1 osd.6 0 waiting for initial osdmap 2024-04-24T22:39:28.155 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-04-24T22:39:28.154+0000 7f37961f6640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:39:28.164 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-04-24T22:39:28.163+0000 7f3d64447640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:39:28.180 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-04-24T22:39:28.179+0000 7f599625d640 -1 osd.4 0 waiting for initial osdmap 2024-04-24T22:39:28.199 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-04-24T22:39:28.197+0000 7f598e88f640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:39:28.396 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-24T22:39:28.395+0000 7f23dc600640 -1 osd.0 0 waiting for initial osdmap 2024-04-24T22:39:28.397 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-24T22:39:28.396+0000 7f701fa00640 -1 osd.3 0 waiting for initial osdmap 2024-04-24T22:39:28.403 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-24T22:39:28.402+0000 7f23d3000640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:39:28.406 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-24T22:39:28.405+0000 7f7016400640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:39:28.417 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-24T22:39:28.416+0000 7fc2d5a00640 -1 osd.1 0 waiting for initial osdmap 2024-04-24T22:39:28.422 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-24T22:39:28.421+0000 7fb79e000640 -1 osd.2 0 waiting for initial osdmap 2024-04-24T22:39:28.429 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-24T22:39:28.428+0000 7fc2ca600640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:39:28.433 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-24T22:39:28.432+0000 7fb794a00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T22:39:29.091 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:29.287 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.286+0000 7f72a3200640 1 Processor -- start 2024-04-24T22:39:29.288 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.286+0000 7f72a3200640 1 -- start start 2024-04-24T22:39:29.289 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.287+0000 7f72a3200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f729c0cc690 0x7f729c1629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.289 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.287+0000 7f72a3200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f729c0c7ea0 con 0x7f729c0c6a50 2024-04-24T22:39:29.289 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f72a3200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f729c0cbbf0 con 0x7f729c158dd0 2024-04-24T22:39:29.289 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f72a3200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f729c162f20 con 0x7f729c0cc690 2024-04-24T22:39:29.289 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f729be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f729c158dd0 0x7f729c0c1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:53988/0 (socket says 172.21.15.47:53988) 2024-04-24T22:39:29.289 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f729be00640 1 -- 172.21.15.47:0/4258031686 learned_addr learned my addr 172.21.15.47:0/4258031686 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:29.289 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f729b400640 1 --2- 172.21.15.47:0/4258031686 >> v2:172.21.15.47:3300/0 conn(0x7f729c0cc690 0x7f729c1629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.290 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f729aa00640 1 -- 172.21.15.47:0/4258031686 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4167286052 0 0) 0x7f729c0cbbf0 con 0x7f729c158dd0 2024-04-24T22:39:29.291 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f729aa00640 1 -- 172.21.15.47:0/4258031686 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7280003520 con 0x7f729c158dd0 2024-04-24T22:39:29.291 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f729aa00640 1 -- 172.21.15.47:0/4258031686 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 134434954 0 0) 0x7f729c0c7ea0 con 0x7f729c0c6a50 2024-04-24T22:39:29.291 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f729aa00640 1 -- 172.21.15.47:0/4258031686 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f729c0cbbf0 con 0x7f729c0c6a50 2024-04-24T22:39:29.291 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f729b400640 1 -- 172.21.15.47:0/4258031686 >> v1:172.21.15.113:6789/0 conn(0x7f729c158dd0 legacy=0x7f729c0c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.291 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f729b400640 1 -- 172.21.15.47:0/4258031686 >> v1:172.21.15.47:6789/0 conn(0x7f729c0c6a50 legacy=0x7f729c0c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.291 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.288+0000 7f729b400640 1 -- 172.21.15.47:0/4258031686 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c1591b0 con 0x7f729c0cc690 2024-04-24T22:39:29.291 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.289+0000 7f729b400640 1 --2- 172.21.15.47:0/4258031686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0cc690 0x7f729c1629e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7288009d90 tx=0x7f728802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b907568e3e18ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:29.292 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.290+0000 7f729aa00640 1 -- 172.21.15.47:0/4258031686 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288007740 con 0x7f729c0cc690 2024-04-24T22:39:29.292 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.290+0000 7f729aa00640 1 -- 172.21.15.47:0/4258031686 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72880078e0 con 0x7f729c0cc690 2024-04-24T22:39:29.292 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.290+0000 7f729aa00640 1 -- 172.21.15.47:0/4258031686 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288007cb0 con 0x7f729c0cc690 2024-04-24T22:39:29.292 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.290+0000 7f72a3200640 1 -- 172.21.15.47:0/4258031686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0cc690 msgr2=0x7f729c1629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.292 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.290+0000 7f72a3200640 1 --2- 172.21.15.47:0/4258031686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0cc690 0x7f729c1629e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7288009d90 tx=0x7f728802a660 comp rx=0 tx=0).stop 2024-04-24T22:39:29.292 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.290+0000 7f72a3200640 1 -- 172.21.15.47:0/4258031686 shutdown_connections 2024-04-24T22:39:29.292 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.290+0000 7f72a3200640 1 --2- 172.21.15.47:0/4258031686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0cc690 0x7f729c1629e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.292 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.290+0000 7f72a3200640 1 -- 172.21.15.47:0/4258031686 >> 172.21.15.47:0/4258031686 conn(0x7f729c067210 msgr2=0x7f729c06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:29.292 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.291+0000 7f72a3200640 1 -- 172.21.15.47:0/4258031686 shutdown_connections 2024-04-24T22:39:29.292 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.291+0000 7f72a3200640 1 -- 172.21.15.47:0/4258031686 wait complete. 2024-04-24T22:39:29.293 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.292+0000 7f72a3200640 1 Processor -- start 2024-04-24T22:39:29.293 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.292+0000 7f72a3200640 1 -- start start 2024-04-24T22:39:29.295 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f72a3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0c6a50 0x7f729c150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.295 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f72a3200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f729c0cc690 0x7f729c154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.295 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f72a3200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729c158dd0 0x7f729c156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.295 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f72a3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f729c0cbe90 con 0x7f729c0c6a50 2024-04-24T22:39:29.295 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f72a3200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f729c0c7bc0 con 0x7f729c158dd0 2024-04-24T22:39:29.295 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f72a3200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f729c0c8cf0 con 0x7f729c0cc690 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f729be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0c6a50 0x7f729c150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f72a1200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729c158dd0 0x7f729c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f729be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0c6a50 0x7f729c150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48068/0 (socket says 172.21.15.47:48068) 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f72a1200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729c158dd0 0x7f729c156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:46496/0 (socket says 172.21.15.47:46496) 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f729b400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f729c0cc690 0x7f729c154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.293+0000 7f729be00640 1 -- 172.21.15.47:0/4186361942 learned_addr learned my addr 172.21.15.47:0/4186361942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.294+0000 7f729be00640 1 -- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f729c0cc690 msgr2=0x7f729c154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.294+0000 7f729be00640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f729c0cc690 0x7f729c154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.294+0000 7f729be00640 1 -- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729c158dd0 msgr2=0x7f729c156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.294+0000 7f729be00640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729c158dd0 0x7f729c156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.294+0000 7f729be00640 1 -- 172.21.15.47:0/4186361942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c14fe90 con 0x7f729c0c6a50 2024-04-24T22:39:29.296 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.294+0000 7f729be00640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0c6a50 0x7f729c150a10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c500 tx=0x7f728c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:29.297 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.294+0000 7f7293e00640 1 -- 172.21.15.47:0/4186361942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c017070 con 0x7f729c0c6a50 2024-04-24T22:39:29.297 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.294+0000 7f7293e00640 1 -- 172.21.15.47:0/4186361942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f728c005d00 con 0x7f729c0c6a50 2024-04-24T22:39:29.297 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.294+0000 7f7293e00640 1 -- 172.21.15.47:0/4186361942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c007330 con 0x7f729c0c6a50 2024-04-24T22:39:29.297 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.294+0000 7f72a3200640 1 -- 172.21.15.47:0/4186361942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729c153260 con 0x7f729c0c6a50 2024-04-24T22:39:29.297 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.295+0000 7f72a3200640 1 -- 172.21.15.47:0/4186361942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729c154dc0 con 0x7f729c0c6a50 2024-04-24T22:39:29.297 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.296+0000 7f72a3200640 1 -- 172.21.15.47:0/4186361942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f729c075910 con 0x7f729c0c6a50 2024-04-24T22:39:29.298 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.296+0000 7f7293e00640 1 -- 172.21.15.47:0/4186361942 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f728c01a070 con 0x7f729c0c6a50 2024-04-24T22:39:29.299 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.298+0000 7f7293e00640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f727c06bda0 0x7f727c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.299 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.298+0000 7f7293e00640 1 -- 172.21.15.47:0/4186361942 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f728c089c80 con 0x7f729c0c6a50 2024-04-24T22:39:29.299 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.298+0000 7f729b400640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f727c06bda0 0x7f727c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.300 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.299+0000 7f729b400640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f727c06bda0 0x7f727c06e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7288034000 tx=0x7f728802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:29.305 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.304+0000 7f7293e00640 1 -- 172.21.15.47:0/4186361942 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f728c058640 con 0x7f729c0c6a50 2024-04-24T22:39:29.500 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.498+0000 7f72a3200640 1 -- 172.21.15.47:0/4186361942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f729c0c75b0 con 0x7f729c0c6a50 2024-04-24T22:39:29.501 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.500+0000 7f7293e00640 1 -- 172.21.15.47:0/4186361942 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10897 (secure 0 0 0) 0x7f728c05c940 con 0x7f729c0c6a50 2024-04-24T22:39:29.501 INFO:teuthology.misc.health.smithi047.stdout: 2024-04-24T22:39:29.501 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":13,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:29.145276+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:29.504 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.502+0000 7f72a3200640 1 -- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f727c06bda0 msgr2=0x7f727c06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.504 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.502+0000 7f72a3200640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f727c06bda0 0x7f727c06e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7288034000 tx=0x7f728802f000 comp rx=0 tx=0).stop 2024-04-24T22:39:29.504 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.502+0000 7f72a3200640 1 -- 172.21.15.47:0/4186361942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0c6a50 msgr2=0x7f729c150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.504 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.502+0000 7f72a3200640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0c6a50 0x7f729c150a10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c500 tx=0x7f728c00c9c0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.504 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.502+0000 7f72a3200640 1 -- 172.21.15.47:0/4186361942 shutdown_connections 2024-04-24T22:39:29.504 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.502+0000 7f72a3200640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f727c06bda0 0x7f727c06e260 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.504 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.502+0000 7f72a3200640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729c158dd0 0x7f729c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.504 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.502+0000 7f72a3200640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f729c0cc690 0x7f729c154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.505 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.502+0000 7f72a3200640 1 --2- 172.21.15.47:0/4186361942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0c6a50 0x7f729c150a10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.505 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.502+0000 7f72a3200640 1 -- 172.21.15.47:0/4186361942 >> 172.21.15.47:0/4186361942 conn(0x7f729c067210 msgr2=0x7f729c0c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:29.505 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.503+0000 7f72a3200640 1 -- 172.21.15.47:0/4186361942 shutdown_connections 2024-04-24T22:39:29.505 INFO:teuthology.misc.health.smithi047.stderr:2024-04-24T22:39:29.503+0000 7f72a3200640 1 -- 172.21.15.47:0/4186361942 wait complete. 2024-04-24T22:39:29.519 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-24T22:39:29.519 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-24T22:39:29.520 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T22:39:29.520 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T22:39:29.528 INFO:tasks.ceph.mds.a:Started 2024-04-24T22:39:29.528 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T22:39:29.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T22:39:29.535 INFO:tasks.ceph.mds.c:Started 2024-04-24T22:39:29.535 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-24T22:39:29.535 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-24T22:39:29.539 INFO:tasks.ceph.mds.e:Started 2024-04-24T22:39:29.539 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T22:39:29.539 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T22:39:29.542 INFO:tasks.ceph.mds.b:Started 2024-04-24T22:39:29.542 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T22:39:29.542 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T22:39:29.545 INFO:tasks.ceph.mds.d:Started 2024-04-24T22:39:29.545 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-24T22:39:29.545 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-24T22:39:29.547 INFO:tasks.ceph.mds.f:Started 2024-04-24T22:39:29.547 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-24T22:39:29.547 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:29.626 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-04-24T22:39:29.627 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-04-24T22:39:29.627 INFO:tasks.ceph.mds.f.smithi113.stdout:starting mds.f at 2024-04-24T22:39:29.672 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-24T22:39:29.677 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-04-24T22:39:29.680 INFO:tasks.ceph.mds.e.smithi047.stdout:starting mds.e at 2024-04-24T22:39:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.720+0000 7fe21ce00640 1 Processor -- start 2024-04-24T22:39:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.720+0000 7fe21ce00640 1 -- start start 2024-04-24T22:39:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe21ce00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe218158fc0 0x7fe2180c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe21ce00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2180c1280 0x7fe2180c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe21ce00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2181621d0 con 0x7fe218158be0 2024-04-24T22:39:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe21ce00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe2181624b0 con 0x7fe2180c1280 2024-04-24T22:39:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe21ce00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe218162790 con 0x7fe218158fc0 2024-04-24T22:39:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe215e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe218158be0 0x7fe2180c78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42896/0 (socket says 172.21.15.47:42896) 2024-04-24T22:39:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe215e00640 1 -- 172.21.15.47:0/691260920 learned_addr learned my addr 172.21.15.47:0/691260920 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe216800640 1 --2- 172.21.15.47:0/691260920 >> v2:172.21.15.47:3300/0 conn(0x7fe2180c1280 0x7fe2180c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe20fe00640 1 --2- 172.21.15.47:0/691260920 >> v2:172.21.15.113:3300/0 conn(0x7fe218158fc0 0x7fe2180c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe215400640 1 -- 172.21.15.47:0/691260920 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 382845086 0 0) 0x7fe2181621d0 con 0x7fe218158be0 2024-04-24T22:39:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe215400640 1 -- 172.21.15.47:0/691260920 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1fc003520 con 0x7fe218158be0 2024-04-24T22:39:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe20fe00640 1 -- 172.21.15.47:0/691260920 >> v2:172.21.15.47:3300/0 conn(0x7fe2180c1280 msgr2=0x7fe2180c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe20fe00640 1 --2- 172.21.15.47:0/691260920 >> v2:172.21.15.47:3300/0 conn(0x7fe2180c1280 0x7fe2180c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe20fe00640 1 -- 172.21.15.47:0/691260920 >> v1:172.21.15.47:6789/0 conn(0x7fe218158be0 legacy=0x7fe2180c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.722+0000 7fe20fe00640 1 -- 172.21.15.47:0/691260920 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe218162a70 con 0x7fe218158fc0 2024-04-24T22:39:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.723+0000 7fe20fe00640 1 --2- 172.21.15.47:0/691260920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218158fc0 0x7fe2180c8590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe2180c76a0 tx=0x7fe20002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a29c08c4da6aca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.724+0000 7fe215400640 1 -- 172.21.15.47:0/691260920 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200037070 con 0x7fe218158fc0 2024-04-24T22:39:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.724+0000 7fe215400640 1 -- 172.21.15.47:0/691260920 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe200032440 con 0x7fe218158fc0 2024-04-24T22:39:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.724+0000 7fe215400640 1 -- 172.21.15.47:0/691260920 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200032810 con 0x7fe218158fc0 2024-04-24T22:39:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.724+0000 7fe21ce00640 1 -- 172.21.15.47:0/691260920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218158fc0 msgr2=0x7fe2180c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.724+0000 7fe21ce00640 1 --2- 172.21.15.47:0/691260920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218158fc0 0x7fe2180c8590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe2180c76a0 tx=0x7fe20002a1a0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.724+0000 7fe21ce00640 1 -- 172.21.15.47:0/691260920 shutdown_connections 2024-04-24T22:39:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.724+0000 7fe21ce00640 1 --2- 172.21.15.47:0/691260920 >> v2:172.21.15.47:3300/0 conn(0x7fe2180c1280 0x7fe2180c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.724+0000 7fe21ce00640 1 --2- 172.21.15.47:0/691260920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218158fc0 0x7fe2180c8590 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.724+0000 7fe21ce00640 1 -- 172.21.15.47:0/691260920 >> 172.21.15.47:0/691260920 conn(0x7fe21806e580 msgr2=0x7fe21806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.724+0000 7fe21ce00640 1 -- 172.21.15.47:0/691260920 shutdown_connections 2024-04-24T22:39:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.725+0000 7fe21ce00640 1 -- 172.21.15.47:0/691260920 wait complete. 2024-04-24T22:39:29.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.726+0000 7fe21ce00640 1 Processor -- start 2024-04-24T22:39:29.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.726+0000 7fe21ce00640 1 -- start start 2024-04-24T22:39:29.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.727+0000 7fe21ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2180c1280 0x7fe2181506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.727+0000 7fe21ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe218158be0 0x7fe218160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.727+0000 7fe21ce00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe218158fc0 0x7fe218160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.727+0000 7fe21ce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe21815ed50 con 0x7fe218158be0 2024-04-24T22:39:29.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.727+0000 7fe21ce00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe21815ebd0 con 0x7fe2180c1280 2024-04-24T22:39:29.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.727+0000 7fe21ce00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe21815eed0 con 0x7fe218158fc0 2024-04-24T22:39:29.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe215e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2180c1280 0x7fe2181506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe216800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe218158fc0 0x7fe218160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe216800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe218158fc0 0x7fe218160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:56938/0 (socket says 172.21.15.47:56938) 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe216800640 1 -- 172.21.15.47:0/1318254404 learned_addr learned my addr 172.21.15.47:0/1318254404 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe20fe00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe218158be0 0x7fe218160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe215e00640 1 -- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe218158fc0 msgr2=0x7fe218160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe215e00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe218158fc0 0x7fe218160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe215e00640 1 -- 172.21.15.47:0/1318254404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe218158be0 msgr2=0x7fe218160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe215e00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe218158be0 0x7fe218160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe215e00640 1 -- 172.21.15.47:0/1318254404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2181f5030 con 0x7fe2180c1280 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.728+0000 7fe215e00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2180c1280 0x7fe2181506c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe20800c430 tx=0x7fe20800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.729+0000 7fe20e000640 1 -- 172.21.15.47:0/1318254404 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208017070 con 0x7fe2180c1280 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.729+0000 7fe20e000640 1 -- 172.21.15.47:0/1318254404 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe208005ce0 con 0x7fe2180c1280 2024-04-24T22:39:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.729+0000 7fe20e000640 1 -- 172.21.15.47:0/1318254404 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2080072f0 con 0x7fe2180c1280 2024-04-24T22:39:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.729+0000 7fe21ce00640 1 -- 172.21.15.47:0/1318254404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2181f5310 con 0x7fe2180c1280 2024-04-24T22:39:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.729+0000 7fe21ce00640 1 -- 172.21.15.47:0/1318254404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe218160150 con 0x7fe2180c1280 2024-04-24T22:39:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.730+0000 7fe21ce00640 1 -- 172.21.15.47:0/1318254404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2180621b0 con 0x7fe2180c1280 2024-04-24T22:39:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.730+0000 7fe20e000640 1 -- 172.21.15.47:0/1318254404 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fe20801a070 con 0x7fe2180c1280 2024-04-24T22:39:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.731+0000 7fe20e000640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe1f406bda0 0x7fe1f406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.731+0000 7fe20e000640 1 -- 172.21.15.47:0/1318254404 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fe208089bc0 con 0x7fe2180c1280 2024-04-24T22:39:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.731+0000 7fe20fe00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe1f406bda0 0x7fe1f406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.732+0000 7fe20fe00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe1f406bda0 0x7fe1f406e260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe200034000 tx=0x7fe20002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.736+0000 7fe20e000640 1 -- 172.21.15.47:0/1318254404 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe2180621b0 con 0x7fe2180c1280 2024-04-24T22:39:29.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.931+0000 7fe21ce00640 1 -- 172.21.15.47:0/1318254404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe218073b90 con 0x7fe2180c1280 2024-04-24T22:39:29.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.933+0000 7fe20e000640 1 -- 172.21.15.47:0/1318254404 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10897 (secure 0 0 0) 0x7fe208007e20 con 0x7fe2180c1280 2024-04-24T22:39:29.934 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:29.935 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":13,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:29.145276+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:29.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.935+0000 7fe21ce00640 1 -- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe1f406bda0 msgr2=0x7fe1f406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.935+0000 7fe21ce00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe1f406bda0 0x7fe1f406e260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe200034000 tx=0x7fe20002e000 comp rx=0 tx=0).stop 2024-04-24T22:39:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.935+0000 7fe21ce00640 1 -- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2180c1280 msgr2=0x7fe2181506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.935+0000 7fe21ce00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2180c1280 0x7fe2181506c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe20800c430 tx=0x7fe20800c8f0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.936+0000 7fe21ce00640 1 -- 172.21.15.47:0/1318254404 shutdown_connections 2024-04-24T22:39:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.936+0000 7fe21ce00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe1f406bda0 0x7fe1f406e260 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.936+0000 7fe21ce00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe218158fc0 0x7fe218160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.936+0000 7fe21ce00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe218158be0 0x7fe218160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.936+0000 7fe21ce00640 1 --2- 172.21.15.47:0/1318254404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2180c1280 0x7fe2181506c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.936+0000 7fe21ce00640 1 -- 172.21.15.47:0/1318254404 >> 172.21.15.47:0/1318254404 conn(0x7fe21806e580 msgr2=0x7fe2180c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.936+0000 7fe21ce00640 1 -- 172.21.15.47:0/1318254404 shutdown_connections 2024-04-24T22:39:29.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:29.936+0000 7fe21ce00640 1 -- 172.21.15.47:0/1318254404 wait complete. 2024-04-24T22:39:29.953 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-24T22:39:29.954 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.143+0000 7f650e200640 1 Processor -- start 2024-04-24T22:39:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.143+0000 7f650e200640 1 -- start start 2024-04-24T22:39:30.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.145+0000 7f650e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 0x7f65080c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:30.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.145+0000 7f650e200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6508073040 0x7f65081411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:30.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.145+0000 7f650e200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f650815c3d0 con 0x7f650806bb90 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.145+0000 7f650e200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6508158bf0 con 0x7f6508073040 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.145+0000 7f650e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f650815c6b0 con 0x7f6508072c60 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.146+0000 7f650d800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f650806bb90 0x7f650815b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:54014/0 (socket says 172.21.15.47:54014) 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.146+0000 7f650d800640 1 -- 172.21.15.47:0/2383508975 learned_addr learned my addr 172.21.15.47:0/2383508975 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.146+0000 7f650ce00640 1 --2- 172.21.15.47:0/2383508975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 0x7f65080c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.146+0000 7f6507e00640 1 --2- 172.21.15.47:0/2383508975 >> v2:172.21.15.47:3300/0 conn(0x7f6508073040 0x7f65081411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.146+0000 7f6507400640 1 -- 172.21.15.47:0/2383508975 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2986023257 0 0) 0x7f650815c3d0 con 0x7f650806bb90 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.146+0000 7f6507400640 1 -- 172.21.15.47:0/2383508975 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64f0003520 con 0x7f650806bb90 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.146+0000 7f650ce00640 1 -- 172.21.15.47:0/2383508975 >> v2:172.21.15.47:3300/0 conn(0x7f6508073040 msgr2=0x7f65081411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.146+0000 7f650ce00640 1 --2- 172.21.15.47:0/2383508975 >> v2:172.21.15.47:3300/0 conn(0x7f6508073040 0x7f65081411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.146+0000 7f650ce00640 1 -- 172.21.15.47:0/2383508975 >> v1:172.21.15.113:6789/0 conn(0x7f650806bb90 legacy=0x7f650815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.146+0000 7f650ce00640 1 -- 172.21.15.47:0/2383508975 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65081585a0 con 0x7f6508072c60 2024-04-24T22:39:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.148+0000 7f650ce00640 1 --2- 172.21.15.47:0/2383508975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 0x7f65080c2d60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0097a0 tx=0x7f64fc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d6c29f5dabf7021 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.148+0000 7f6507400640 1 -- 172.21.15.47:0/2383508975 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc037070 con 0x7f6508072c60 2024-04-24T22:39:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.149+0000 7f6507400640 1 -- 172.21.15.47:0/2383508975 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64fc032440 con 0x7f6508072c60 2024-04-24T22:39:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.149+0000 7f6507400640 1 -- 172.21.15.47:0/2383508975 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc032810 con 0x7f6508072c60 2024-04-24T22:39:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.149+0000 7f650e200640 1 -- 172.21.15.47:0/2383508975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 msgr2=0x7f65080c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.149+0000 7f650e200640 1 --2- 172.21.15.47:0/2383508975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 0x7f65080c2d60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0097a0 tx=0x7f64fc02a210 comp rx=0 tx=0).stop 2024-04-24T22:39:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.150+0000 7f650e200640 1 -- 172.21.15.47:0/2383508975 shutdown_connections 2024-04-24T22:39:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.150+0000 7f650e200640 1 --2- 172.21.15.47:0/2383508975 >> v2:172.21.15.47:3300/0 conn(0x7f6508073040 0x7f65081411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.150+0000 7f650e200640 1 --2- 172.21.15.47:0/2383508975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 0x7f65080c2d60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.150+0000 7f650e200640 1 -- 172.21.15.47:0/2383508975 >> 172.21.15.47:0/2383508975 conn(0x7f65080cb900 msgr2=0x7f65080cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:30.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.150+0000 7f650e200640 1 -- 172.21.15.47:0/2383508975 shutdown_connections 2024-04-24T22:39:30.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.150+0000 7f650e200640 1 -- 172.21.15.47:0/2383508975 wait complete. 2024-04-24T22:39:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.152+0000 7f650e200640 1 Processor -- start 2024-04-24T22:39:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.152+0000 7f650e200640 1 -- start start 2024-04-24T22:39:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f650e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650806bb90 0x7f65081e2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f650e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 0x7f65081e2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f650e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6508073040 0x7f65081f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f650e200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65080c32a0 con 0x7f650806bb90 2024-04-24T22:39:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f650e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6508158270 con 0x7f6508073040 2024-04-24T22:39:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f650e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6508159080 con 0x7f6508072c60 2024-04-24T22:39:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f650ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650806bb90 0x7f65081e2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f6507e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 0x7f65081e2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f650d800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6508073040 0x7f65081f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f650ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650806bb90 0x7f65081e2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48132/0 (socket says 172.21.15.47:48132) 2024-04-24T22:39:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.153+0000 7f650ce00640 1 -- 172.21.15.47:0/490679506 learned_addr learned my addr 172.21.15.47:0/490679506 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.154+0000 7f650ce00640 1 -- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 msgr2=0x7f65081e2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.154+0000 7f650ce00640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 0x7f65081e2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.154+0000 7f650ce00640 1 -- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6508073040 msgr2=0x7f65081f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.154+0000 7f650ce00640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6508073040 0x7f65081f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.154+0000 7f650ce00640 1 -- 172.21.15.47:0/490679506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6508159740 con 0x7f650806bb90 2024-04-24T22:39:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.154+0000 7f650ce00640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650806bb90 0x7f65081e2840 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f64fc035040 tx=0x7f64fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.155+0000 7f6504c00640 1 -- 172.21.15.47:0/490679506 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc041030 con 0x7f650806bb90 2024-04-24T22:39:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.155+0000 7f6504c00640 1 -- 172.21.15.47:0/490679506 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64fc03d4e0 con 0x7f650806bb90 2024-04-24T22:39:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.155+0000 7f6504c00640 1 -- 172.21.15.47:0/490679506 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc037040 con 0x7f650806bb90 2024-04-24T22:39:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.155+0000 7f650e200640 1 -- 172.21.15.47:0/490679506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6508159a20 con 0x7f650806bb90 2024-04-24T22:39:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.155+0000 7f650e200640 1 -- 172.21.15.47:0/490679506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65080715c0 con 0x7f650806bb90 2024-04-24T22:39:30.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.156+0000 7f650e200640 1 -- 172.21.15.47:0/490679506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6508068d50 con 0x7f650806bb90 2024-04-24T22:39:30.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.157+0000 7f6504c00640 1 -- 172.21.15.47:0/490679506 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f64fc044070 con 0x7f650806bb90 2024-04-24T22:39:30.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.158+0000 7f6504c00640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06bda0 0x7f64ec06e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:30.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.158+0000 7f6504c00640 1 -- 172.21.15.47:0/490679506 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4471+0+0 (secure 0 0 0) 0x7f64fc0aa9c0 con 0x7f650806bb90 2024-04-24T22:39:30.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.158+0000 7f6507e00640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06bda0 0x7f64ec06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:30.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.159+0000 7f6507e00640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06bda0 0x7f64ec06e260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6508157ff0 tx=0x7f64f8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:30.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.163+0000 7f6504c00640 1 -- 172.21.15.47:0/490679506 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f64fc0791b0 con 0x7f650806bb90 2024-04-24T22:39:30.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.397+0000 7f650e200640 1 -- 172.21.15.47:0/490679506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f650804f5e0 con 0x7f650806bb90 2024-04-24T22:39:30.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.398+0000 7f6504c00640 1 -- 172.21.15.47:0/490679506 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f64fc07d4b0 con 0x7f650806bb90 2024-04-24T22:39:30.400 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:30.400 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3103107210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3103107210},{"type":"v1","addr":"172.21.15.113:6837","nonce":3103107210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T22:39:30.400 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 2 2024-04-24T22:39:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 -- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06bda0 msgr2=0x7f64ec06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06bda0 0x7f64ec06e260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6508157ff0 tx=0x7f64f8009190 comp rx=0 tx=0).stop 2024-04-24T22:39:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 -- 172.21.15.47:0/490679506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650806bb90 msgr2=0x7f65081e2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:30.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650806bb90 0x7f65081e2840 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f64fc035040 tx=0x7f64fc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 -- 172.21.15.47:0/490679506 shutdown_connections 2024-04-24T22:39:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06bda0 0x7f64ec06e260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6508073040 0x7f65081f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6508072c60 0x7f65081e2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 --2- 172.21.15.47:0/490679506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650806bb90 0x7f65081e2840 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 -- 172.21.15.47:0/490679506 >> 172.21.15.47:0/490679506 conn(0x7f65080cb900 msgr2=0x7f65081575b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 -- 172.21.15.47:0/490679506 shutdown_connections 2024-04-24T22:39:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:30.401+0000 7f650e200640 1 -- 172.21.15.47:0/490679506 wait complete. 2024-04-24T22:39:32.419 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:32.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.633+0000 7fd009400640 1 Processor -- start 2024-04-24T22:39:32.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.634+0000 7fd009400640 1 -- start start 2024-04-24T22:39:32.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.636+0000 7fd009400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd00406bb90 0x7fd00415b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:32.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.636+0000 7fd009400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00415c3d0 con 0x7fd004072c60 2024-04-24T22:39:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.636+0000 7fd009400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00415c6b0 con 0x7fd004073040 2024-04-24T22:39:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.636+0000 7fd009400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd004158bf0 con 0x7fd00406bb90 2024-04-24T22:39:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.636+0000 7fd003e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd004072c60 0x7fd0040c2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42898/0 (socket says 172.21.15.47:42898) 2024-04-24T22:39:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.636+0000 7fd003e00640 1 -- 172.21.15.47:0/3853839836 learned_addr learned my addr 172.21.15.47:0/3853839836 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.636+0000 7fd008a00640 1 --2- 172.21.15.47:0/3853839836 >> v2:172.21.15.113:3300/0 conn(0x7fd00406bb90 0x7fd00415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.636+0000 7fd003400640 1 -- 172.21.15.47:0/3853839836 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 314233858 0 0) 0x7fd00415c3d0 con 0x7fd004072c60 2024-04-24T22:39:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.636+0000 7fd003400640 1 -- 172.21.15.47:0/3853839836 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfe8003520 con 0x7fd004072c60 2024-04-24T22:39:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.637+0000 7fd008a00640 1 -- 172.21.15.47:0/3853839836 >> v1:172.21.15.113:6789/0 conn(0x7fd004073040 legacy=0x7fd0041411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.637+0000 7fd008a00640 1 -- 172.21.15.47:0/3853839836 >> v1:172.21.15.47:6789/0 conn(0x7fd004072c60 legacy=0x7fd0040c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.637+0000 7fd003400640 1 -- 172.21.15.47:0/3853839836 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1886607858 0 0) 0x7fd00415c6b0 con 0x7fd004073040 2024-04-24T22:39:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.637+0000 7fd008a00640 1 -- 172.21.15.47:0/3853839836 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0041585a0 con 0x7fd00406bb90 2024-04-24T22:39:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.638+0000 7fd008a00640 1 --2- 172.21.15.47:0/3853839836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00406bb90 0x7fd00415b860 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009d90 tx=0x7fcffc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d17989933e8ba67f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.638+0000 7fd003400640 1 -- 172.21.15.47:0/3853839836 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc007740 con 0x7fd00406bb90 2024-04-24T22:39:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.638+0000 7fd003400640 1 -- 172.21.15.47:0/3853839836 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcffc0078e0 con 0x7fd00406bb90 2024-04-24T22:39:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.638+0000 7fd003400640 1 -- 172.21.15.47:0/3853839836 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc007cb0 con 0x7fd00406bb90 2024-04-24T22:39:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.638+0000 7fd009400640 1 -- 172.21.15.47:0/3853839836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00406bb90 msgr2=0x7fd00415b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:32.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.638+0000 7fd009400640 1 --2- 172.21.15.47:0/3853839836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00406bb90 0x7fd00415b860 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009d90 tx=0x7fcffc02a6f0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.639+0000 7fd009400640 1 -- 172.21.15.47:0/3853839836 shutdown_connections 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.639+0000 7fd009400640 1 --2- 172.21.15.47:0/3853839836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00406bb90 0x7fd00415b860 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.639+0000 7fd009400640 1 -- 172.21.15.47:0/3853839836 >> 172.21.15.47:0/3853839836 conn(0x7fd0040cb900 msgr2=0x7fd0040cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.639+0000 7fd009400640 1 -- 172.21.15.47:0/3853839836 shutdown_connections 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.639+0000 7fd009400640 1 -- 172.21.15.47:0/3853839836 wait complete. 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.640+0000 7fd009400640 1 Processor -- start 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.641+0000 7fd009400640 1 -- start start 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.641+0000 7fd009400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00406bb90 0x7fd0041f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fd009400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd004072c60 0x7fd004071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fd009400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd004073040 0x7fd00413d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fd009400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd004159080 con 0x7fd004072c60 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fd009400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd004158270 con 0x7fd00406bb90 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fd009400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd004159200 con 0x7fd004073040 2024-04-24T22:39:32.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fcffbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd004072c60 0x7fd004071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fd008a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd004073040 0x7fd00413d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fd003e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00406bb90 0x7fd0041f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fcffbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd004072c60 0x7fd004071bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48138/0 (socket says 172.21.15.47:48138) 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fcffbe00640 1 -- 172.21.15.47:0/1553958567 learned_addr learned my addr 172.21.15.47:0/1553958567 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fd008a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd004073040 0x7fd00413d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:56958/0 (socket says 172.21.15.47:56958) 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fcffbe00640 1 -- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd004073040 msgr2=0x7fd00413d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fcffbe00640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd004073040 0x7fd00413d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fcffbe00640 1 -- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00406bb90 msgr2=0x7fd0041f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fcffbe00640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00406bb90 0x7fd0041f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.642+0000 7fcffbe00640 1 -- 172.21.15.47:0/1553958567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00413e510 con 0x7fd004072c60 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.643+0000 7fcffbe00640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd004072c60 0x7fd004071bd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00c4e0 tx=0x7fcfec00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.643+0000 7fd000c00640 1 -- 172.21.15.47:0/1553958567 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec017070 con 0x7fd004072c60 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.643+0000 7fd000c00640 1 -- 172.21.15.47:0/1553958567 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec0035f0 con 0x7fd004072c60 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.643+0000 7fd009400640 1 -- 172.21.15.47:0/1553958567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0040c88c0 con 0x7fd004072c60 2024-04-24T22:39:32.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.643+0000 7fd000c00640 1 -- 172.21.15.47:0/1553958567 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0039e0 con 0x7fd004072c60 2024-04-24T22:39:32.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.643+0000 7fd009400640 1 -- 172.21.15.47:0/1553958567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0041f37e0 con 0x7fd004072c60 2024-04-24T22:39:32.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.644+0000 7fd009400640 1 -- 172.21.15.47:0/1553958567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0040c52f0 con 0x7fd004072c60 2024-04-24T22:39:32.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.646+0000 7fd000c00640 1 -- 172.21.15.47:0/1553958567 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fcfec01a070 con 0x7fd004072c60 2024-04-24T22:39:32.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.647+0000 7fd000c00640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcfe006bda0 0x7fcfe006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:32.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.647+0000 7fd000c00640 1 -- 172.21.15.47:0/1553958567 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4846+0+0 (secure 0 0 0) 0x7fcfec08a4d0 con 0x7fd004072c60 2024-04-24T22:39:32.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.648+0000 7fd003e00640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcfe006bda0 0x7fcfe006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:32.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.648+0000 7fd003e00640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcfe006bda0 0x7fcfe006e260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcff4005880 tx=0x7fcff4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.654+0000 7fd000c00640 1 -- 172.21.15.47:0/1553958567 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd0040c52f0 con 0x7fd004072c60 2024-04-24T22:39:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.920+0000 7fd009400640 1 -- 172.21.15.47:0/1553958567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd004068d50 con 0x7fd004072c60 2024-04-24T22:39:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.921+0000 7fd000c00640 1 -- 172.21.15.47:0/1553958567 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5521 (secure 0 0 0) 0x7fcfec058cc0 con 0x7fd004072c60 2024-04-24T22:39:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:32.923 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1712577517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1712577517},{"type":"v1","addr":"172.21.15.113:6839","nonce":1712577517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2908847255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2908847255},{"type":"v1","addr":"172.21.15.47:6833","nonce":2908847255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2173854280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2173854280},{"type":"v1","addr":"172.21.15.47:6837","nonce":2173854280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3103107210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3103107210},{"type":"v1","addr":"172.21.15.113:6837","nonce":3103107210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6838/3621458364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3621458364},{"type":"v1","addr":"172.21.15.113:6838","nonce":3621458364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1117134154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1117134154},{"type":"v1","addr":"172.21.15.47:6835","nonce":1117134154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T22:39:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-04-24T22:39:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.924+0000 7fd009400640 1 -- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcfe006bda0 msgr2=0x7fcfe006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.924+0000 7fd009400640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcfe006bda0 0x7fcfe006e260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcff4005880 tx=0x7fcff4009040 comp rx=0 tx=0).stop 2024-04-24T22:39:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.924+0000 7fd009400640 1 -- 172.21.15.47:0/1553958567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd004072c60 msgr2=0x7fd004071bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.924+0000 7fd009400640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd004072c60 0x7fd004071bd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00c4e0 tx=0x7fcfec00c9a0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.925+0000 7fd009400640 1 -- 172.21.15.47:0/1553958567 shutdown_connections 2024-04-24T22:39:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.925+0000 7fd009400640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcfe006bda0 0x7fcfe006e260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.925+0000 7fd009400640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd004073040 0x7fd00413d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.925+0000 7fd009400640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd004072c60 0x7fd004071bd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.925+0000 7fd009400640 1 --2- 172.21.15.47:0/1553958567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd00406bb90 0x7fd0041f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.925+0000 7fd009400640 1 -- 172.21.15.47:0/1553958567 >> 172.21.15.47:0/1553958567 conn(0x7fd0040cb900 msgr2=0x7fd0040cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.925+0000 7fd009400640 1 -- 172.21.15.47:0/1553958567 shutdown_connections 2024-04-24T22:39:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:32.925+0000 7fd009400640 1 -- 172.21.15.47:0/1553958567 wait complete. 2024-04-24T22:39:32.948 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:33.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.140+0000 7f9f2ac00640 1 Processor -- start 2024-04-24T22:39:33.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.140+0000 7f9f2ac00640 1 -- start start 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.141+0000 7f9f2ac00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f2415e4f0 0x7f9f2406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f2ac00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2406b4a0 con 0x7f9f241546a0 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f2ac00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2406b740 con 0x7f9f2415d9f0 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f2ac00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9f240c5010 con 0x7f9f2415e4f0 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f28e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f2415e4f0 0x7f9f2406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f29800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9f241546a0 0x7f9f240c5d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42910/0 (socket says 172.21.15.47:42910) 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f29800640 1 -- 172.21.15.47:0/2532921722 learned_addr learned my addr 172.21.15.47:0/2532921722 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f23e00640 1 -- 172.21.15.47:0/2532921722 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4096582632 0 0) 0x7f9f2406b740 con 0x7f9f2415d9f0 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f23e00640 1 -- 172.21.15.47:0/2532921722 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f0c003520 con 0x7f9f2415d9f0 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f28e00640 1 -- 172.21.15.47:0/2532921722 >> v1:172.21.15.113:6789/0 conn(0x7f9f2415d9f0 legacy=0x7f9f24164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f28e00640 1 -- 172.21.15.47:0/2532921722 >> v1:172.21.15.47:6789/0 conn(0x7f9f241546a0 legacy=0x7f9f240c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f23e00640 1 -- 172.21.15.47:0/2532921722 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 875386685 0 0) 0x7f9f2406b4a0 con 0x7f9f241546a0 2024-04-24T22:39:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.142+0000 7f9f28e00640 1 -- 172.21.15.47:0/2532921722 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f240c5260 con 0x7f9f2415e4f0 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.144+0000 7f9f28e00640 1 --2- 172.21.15.47:0/2532921722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415e4f0 0x7f9f2406c9c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9f240c5190 tx=0x7f9f1002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9124b451bf66d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.144+0000 7f9f23e00640 1 -- 172.21.15.47:0/2532921722 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10037070 con 0x7f9f2415e4f0 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.144+0000 7f9f23e00640 1 -- 172.21.15.47:0/2532921722 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f10032440 con 0x7f9f2415e4f0 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.144+0000 7f9f23e00640 1 -- 172.21.15.47:0/2532921722 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10032810 con 0x7f9f2415e4f0 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.144+0000 7f9f2ac00640 1 -- 172.21.15.47:0/2532921722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415e4f0 msgr2=0x7f9f2406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.145+0000 7f9f2ac00640 1 --2- 172.21.15.47:0/2532921722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415e4f0 0x7f9f2406c9c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9f240c5190 tx=0x7f9f1002a1c0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.145+0000 7f9f2ac00640 1 -- 172.21.15.47:0/2532921722 shutdown_connections 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.145+0000 7f9f2ac00640 1 --2- 172.21.15.47:0/2532921722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415e4f0 0x7f9f2406c9c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.145+0000 7f9f2ac00640 1 -- 172.21.15.47:0/2532921722 >> 172.21.15.47:0/2532921722 conn(0x7f9f24072d20 msgr2=0x7f9f240cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.145+0000 7f9f2ac00640 1 -- 172.21.15.47:0/2532921722 shutdown_connections 2024-04-24T22:39:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.145+0000 7f9f2ac00640 1 -- 172.21.15.47:0/2532921722 wait complete. 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.146+0000 7f9f2ac00640 1 Processor -- start 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.147+0000 7f9f2ac00640 1 -- start start 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.147+0000 7f9f2ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f241546a0 0x7f9f24141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.147+0000 7f9f2ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415d9f0 0x7f9f241e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f2ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f2415e4f0 0x7f9f24144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f2ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2406b160 con 0x7f9f2415d9f0 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f2ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24154a80 con 0x7f9f2415e4f0 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f2ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9f240ca630 con 0x7f9f241546a0 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f28e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415d9f0 0x7f9f241e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f29800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f241546a0 0x7f9f24141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f28e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415d9f0 0x7f9f241e48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48158/0 (socket says 172.21.15.47:48158) 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f28e00640 1 -- 172.21.15.47:0/3296279690 learned_addr learned my addr 172.21.15.47:0/3296279690 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f2a200640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f2415e4f0 0x7f9f24144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f28e00640 1 -- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f241546a0 msgr2=0x7f9f24141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f28e00640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f241546a0 0x7f9f24141b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f28e00640 1 -- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f2415e4f0 msgr2=0x7f9f24144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f28e00640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f2415e4f0 0x7f9f24144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.148+0000 7f9f28e00640 1 -- 172.21.15.47:0/3296279690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f24142740 con 0x7f9f2415d9f0 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.149+0000 7f9f28e00640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415d9f0 0x7f9f241e48c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9f1002a7b0 tx=0x7f9f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.149+0000 7f9f21600640 1 -- 172.21.15.47:0/3296279690 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10041070 con 0x7f9f2415d9f0 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.149+0000 7f9f21600640 1 -- 172.21.15.47:0/3296279690 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1003c890 con 0x7f9f2415d9f0 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.149+0000 7f9f2ac00640 1 -- 172.21.15.47:0/3296279690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f241429e0 con 0x7f9f2415d9f0 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.149+0000 7f9f21600640 1 -- 172.21.15.47:0/3296279690 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10037040 con 0x7f9f2415d9f0 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.149+0000 7f9f2ac00640 1 -- 172.21.15.47:0/3296279690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f24146210 con 0x7f9f2415d9f0 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.150+0000 7f9f2ac00640 1 -- 172.21.15.47:0/3296279690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f24068520 con 0x7f9f2415d9f0 2024-04-24T22:39:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.151+0000 7f9f21600640 1 -- 172.21.15.47:0/3296279690 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f9f10044070 con 0x7f9f2415d9f0 2024-04-24T22:39:33.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.153+0000 7f9f21600640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9f0406bdd0 0x7f9f0406e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.153+0000 7f9f21600640 1 -- 172.21.15.47:0/3296279690 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9f100aafe0 con 0x7f9f2415d9f0 2024-04-24T22:39:33.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.153+0000 7f9f29800640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9f0406bdd0 0x7f9f0406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.154+0000 7f9f29800640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9f0406bdd0 0x7f9f0406e290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005880 tx=0x7f9f18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:33.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.157+0000 7f9f21600640 1 -- 172.21.15.47:0/3296279690 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9f10079700 con 0x7f9f2415d9f0 2024-04-24T22:39:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.371+0000 7f9f2ac00640 1 -- 172.21.15.47:0/3296279690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f24157e20 con 0x7f9f2415d9f0 2024-04-24T22:39:33.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.372+0000 7f9f21600640 1 -- 172.21.15.47:0/3296279690 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12885 (secure 0 0 0) 0x7f9f1007da00 con 0x7f9f2415d9f0 2024-04-24T22:39:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:33.375 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":16,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:32.644086+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 -- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9f0406bdd0 msgr2=0x7f9f0406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9f0406bdd0 0x7f9f0406e290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005880 tx=0x7f9f18009040 comp rx=0 tx=0).stop 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 -- 172.21.15.47:0/3296279690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415d9f0 msgr2=0x7f9f241e48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415d9f0 0x7f9f241e48c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9f1002a7b0 tx=0x7f9f100046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 -- 172.21.15.47:0/3296279690 shutdown_connections 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9f0406bdd0 0x7f9f0406e290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f2415e4f0 0x7f9f24144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2415d9f0 0x7f9f241e48c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 --2- 172.21.15.47:0/3296279690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f241546a0 0x7f9f24141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 -- 172.21.15.47:0/3296279690 >> 172.21.15.47:0/3296279690 conn(0x7f9f24072d20 msgr2=0x7f9f24143540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 -- 172.21.15.47:0/3296279690 shutdown_connections 2024-04-24T22:39:33.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.375+0000 7f9f2ac00640 1 -- 172.21.15.47:0/3296279690 wait complete. 2024-04-24T22:39:33.396 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:30.711538+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:39:33.397 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:39:33.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.586+0000 7f8251800640 1 Processor -- start 2024-04-24T22:39:33.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.587+0000 7f8251800640 1 -- start start 2024-04-24T22:39:33.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.588+0000 7f8251800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f824c0cbde0 0x7f824c0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.588+0000 7f8251800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f824c15b240 con 0x7f824c073380 2024-04-24T22:39:33.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.588+0000 7f8251800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f824c15dcf0 con 0x7f824c0cc840 2024-04-24T22:39:33.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.588+0000 7f8251800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f824c15daa0 con 0x7f824c0cbde0 2024-04-24T22:39:33.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.589+0000 7f8243e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f824c0cbde0 0x7f824c0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.589+0000 7f824aa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f824c0cc840 0x7f824c0cb6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:54038/0 (socket says 172.21.15.47:54038) 2024-04-24T22:39:33.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.589+0000 7f824aa00640 1 -- 172.21.15.47:0/2233084594 learned_addr learned my addr 172.21.15.47:0/2233084594 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:33.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.589+0000 7f824b400640 1 --1- 172.21.15.47:0/2233084594 >> v1:172.21.15.47:6789/0 conn(0x7f824c073380 0x7f824c164e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42924/0 (socket says 172.21.15.47:42924) 2024-04-24T22:39:33.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.589+0000 7f824a000640 1 -- 172.21.15.47:0/2233084594 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220319855 0 0) 0x7f824c15dcf0 con 0x7f824c0cc840 2024-04-24T22:39:33.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.589+0000 7f824a000640 1 -- 172.21.15.47:0/2233084594 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8230003520 con 0x7f824c0cc840 2024-04-24T22:39:33.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.589+0000 7f8243e00640 1 -- 172.21.15.47:0/2233084594 >> v1:172.21.15.113:6789/0 conn(0x7f824c0cc840 legacy=0x7f824c0cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.589+0000 7f8243e00640 1 -- 172.21.15.47:0/2233084594 >> v1:172.21.15.47:6789/0 conn(0x7f824c073380 legacy=0x7f824c164e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.589+0000 7f824a000640 1 -- 172.21.15.47:0/2233084594 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 661515299 0 0) 0x7f824c15b240 con 0x7f824c073380 2024-04-24T22:39:33.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.589+0000 7f8243e00640 1 -- 172.21.15.47:0/2233084594 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c15dfd0 con 0x7f824c0cbde0 2024-04-24T22:39:33.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.590+0000 7f8243e00640 1 --2- 172.21.15.47:0/2233084594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 0x7f824c0ca7d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f824c15dc20 tx=0x7f823402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5736e10bcb079187 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:33.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.590+0000 7f824a000640 1 -- 172.21.15.47:0/2233084594 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234037070 con 0x7f824c0cbde0 2024-04-24T22:39:33.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.591+0000 7f824a000640 1 -- 172.21.15.47:0/2233084594 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8234032440 con 0x7f824c0cbde0 2024-04-24T22:39:33.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.591+0000 7f824a000640 1 -- 172.21.15.47:0/2233084594 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234032810 con 0x7f824c0cbde0 2024-04-24T22:39:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.591+0000 7f8251800640 1 -- 172.21.15.47:0/2233084594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 msgr2=0x7f824c0ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.591+0000 7f8251800640 1 --2- 172.21.15.47:0/2233084594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 0x7f824c0ca7d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f824c15dc20 tx=0x7f823402a180 comp rx=0 tx=0).stop 2024-04-24T22:39:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.591+0000 7f8251800640 1 -- 172.21.15.47:0/2233084594 shutdown_connections 2024-04-24T22:39:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.591+0000 7f8251800640 1 --2- 172.21.15.47:0/2233084594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 0x7f824c0ca7d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.591+0000 7f8251800640 1 -- 172.21.15.47:0/2233084594 >> 172.21.15.47:0/2233084594 conn(0x7f824c06c940 msgr2=0x7f824c072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.591+0000 7f8251800640 1 -- 172.21.15.47:0/2233084594 shutdown_connections 2024-04-24T22:39:33.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.591+0000 7f8251800640 1 -- 172.21.15.47:0/2233084594 wait complete. 2024-04-24T22:39:33.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.593+0000 7f8251800640 1 Processor -- start 2024-04-24T22:39:33.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.593+0000 7f8251800640 1 -- start start 2024-04-24T22:39:33.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f8251800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f824c073380 0x7f824c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f8251800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 0x7f824c0728d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f8251800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f824c0cc840 0x7f824c14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f8251800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f824c15e870 con 0x7f824c0cbde0 2024-04-24T22:39:33.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f8251800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f824c15e590 con 0x7f824c0cc840 2024-04-24T22:39:33.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f8251800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f824c15af30 con 0x7f824c073380 2024-04-24T22:39:33.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f8243e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 0x7f824c0728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f8243e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 0x7f824c0728d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48166/0 (socket says 172.21.15.47:48166) 2024-04-24T22:39:33.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f8243e00640 1 -- 172.21.15.47:0/3445644034 learned_addr learned my addr 172.21.15.47:0/3445644034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:33.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f824b400640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f824c0cc840 0x7f824c14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.594+0000 7f824aa00640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f824c073380 0x7f824c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.595+0000 7f8243e00640 1 -- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f824c073380 msgr2=0x7f824c1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.595+0000 7f8243e00640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f824c073380 0x7f824c1599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.595+0000 7f8243e00640 1 -- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f824c0cc840 msgr2=0x7f824c14c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.595+0000 7f8243e00640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f824c0cc840 0x7f824c14c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.595+0000 7f8243e00640 1 -- 172.21.15.47:0/3445644034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c1e3d00 con 0x7f824c0cbde0 2024-04-24T22:39:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.595+0000 7f8243e00640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 0x7f824c0728d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8234009890 tx=0x7f82340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.595+0000 7f8242a00640 1 -- 172.21.15.47:0/3445644034 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234041020 con 0x7f824c0cbde0 2024-04-24T22:39:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.595+0000 7f8242a00640 1 -- 172.21.15.47:0/3445644034 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823403d6e0 con 0x7f824c0cbde0 2024-04-24T22:39:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.595+0000 7f8242a00640 1 -- 172.21.15.47:0/3445644034 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234037040 con 0x7f824c0cbde0 2024-04-24T22:39:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.596+0000 7f8251800640 1 -- 172.21.15.47:0/3445644034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c159320 con 0x7f824c0cbde0 2024-04-24T22:39:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.596+0000 7f8251800640 1 -- 172.21.15.47:0/3445644034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c1e53a0 con 0x7f824c0cbde0 2024-04-24T22:39:33.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.597+0000 7f8251800640 1 -- 172.21.15.47:0/3445644034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f824c073b90 con 0x7f824c0cbde0 2024-04-24T22:39:33.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.598+0000 7f8242a00640 1 -- 172.21.15.47:0/3445644034 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f8234044020 con 0x7f824c0cbde0 2024-04-24T22:39:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.599+0000 7f8242a00640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f822806bda0 0x7f822806e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.599+0000 7f8242a00640 1 -- 172.21.15.47:0/3445644034 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f82340ab090 con 0x7f824c0cbde0 2024-04-24T22:39:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.599+0000 7f824aa00640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f822806bda0 0x7f822806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.600+0000 7f824aa00640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f822806bda0 0x7f822806e260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f823c005880 tx=0x7f823c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.604+0000 7f8242a00640 1 -- 172.21.15.47:0/3445644034 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f823404b030 con 0x7f824c0cbde0 2024-04-24T22:39:33.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.811+0000 7f8251800640 1 -- 172.21.15.47:0/3445644034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f824c15a900 con 0x7f824c0cbde0 2024-04-24T22:39:33.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.811+0000 7f8242a00640 1 -- 172.21.15.47:0/3445644034 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f8234079820 con 0x7f824c0cbde0 2024-04-24T22:39:33.814 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-24T22:39:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.814+0000 7f8251800640 1 -- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f822806bda0 msgr2=0x7f822806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.814+0000 7f8251800640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f822806bda0 0x7f822806e260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f823c005880 tx=0x7f823c009040 comp rx=0 tx=0).stop 2024-04-24T22:39:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.814+0000 7f8251800640 1 -- 172.21.15.47:0/3445644034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 msgr2=0x7f824c0728d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.814+0000 7f8251800640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 0x7f824c0728d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8234009890 tx=0x7f82340046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.814+0000 7f8251800640 1 -- 172.21.15.47:0/3445644034 shutdown_connections 2024-04-24T22:39:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.814+0000 7f8251800640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f822806bda0 0x7f822806e260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.814+0000 7f8251800640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f824c0cc840 0x7f824c14c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.814+0000 7f8251800640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f824c0cbde0 0x7f824c0728d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.814+0000 7f8251800640 1 --2- 172.21.15.47:0/3445644034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f824c073380 0x7f824c1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.814+0000 7f8251800640 1 -- 172.21.15.47:0/3445644034 >> 172.21.15.47:0/3445644034 conn(0x7f824c06c940 msgr2=0x7f824c0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.815+0000 7f8251800640 1 -- 172.21.15.47:0/3445644034 shutdown_connections 2024-04-24T22:39:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:33.815+0000 7f8251800640 1 -- 172.21.15.47:0/3445644034 wait complete. 2024-04-24T22:39:33.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T22:39:34.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.043+0000 7f9ebf200640 1 Processor -- start 2024-04-24T22:39:34.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.044+0000 7f9ebf200640 1 -- start start 2024-04-24T22:39:34.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.045+0000 7f9ebf200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb806e1b0 0x7f9eb806e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.045+0000 7f9ebf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9eb80707e0 0x7f9eb815e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.045+0000 7f9ebf200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb80c6f30 con 0x7f9eb8070ea0 2024-04-24T22:39:34.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.045+0000 7f9ebf200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb80c7210 con 0x7f9eb80707e0 2024-04-24T22:39:34.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.045+0000 7f9ebf200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb815a500 con 0x7f9eb806e1b0 2024-04-24T22:39:34.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.046+0000 7f9ebde00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb806e1b0 0x7f9eb806e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.046+0000 7f9ebde00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb806e1b0 0x7f9eb806e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:56982/0 (socket says 172.21.15.47:56982) 2024-04-24T22:39:34.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.046+0000 7f9ebde00640 1 -- 172.21.15.47:0/1241728534 learned_addr learned my addr 172.21.15.47:0/1241728534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:34.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.046+0000 7f9ebd400640 1 --2- 172.21.15.47:0/1241728534 >> v2:172.21.15.113:3300/0 conn(0x7f9eb80707e0 0x7f9eb815e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.046+0000 7f9ebde00640 1 -- 172.21.15.47:0/1241728534 >> v2:172.21.15.113:3300/0 conn(0x7f9eb80707e0 msgr2=0x7f9eb815e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:34.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.046+0000 7f9ebde00640 1 --2- 172.21.15.47:0/1241728534 >> v2:172.21.15.113:3300/0 conn(0x7f9eb80707e0 0x7f9eb815e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.046+0000 7f9ebde00640 1 -- 172.21.15.47:0/1241728534 >> v1:172.21.15.47:6789/0 conn(0x7f9eb8070ea0 legacy=0x7f9eb806b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:34.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.046+0000 7f9ebde00640 1 -- 172.21.15.47:0/1241728534 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb815a7e0 con 0x7f9eb806e1b0 2024-04-24T22:39:34.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.047+0000 7f9ebde00640 1 --2- 172.21.15.47:0/1241728534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb806e1b0 0x7f9eb806e590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009870 tx=0x7f9eac02a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd53a0a925664ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:34.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.047+0000 7f9ebca00640 1 -- 172.21.15.47:0/1241728534 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac037070 con 0x7f9eb806e1b0 2024-04-24T22:39:34.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.047+0000 7f9ebca00640 1 -- 172.21.15.47:0/1241728534 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eac032440 con 0x7f9eb806e1b0 2024-04-24T22:39:34.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.047+0000 7f9ebca00640 1 -- 172.21.15.47:0/1241728534 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac032810 con 0x7f9eb806e1b0 2024-04-24T22:39:34.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.048+0000 7f9ebf200640 1 -- 172.21.15.47:0/1241728534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb806e1b0 msgr2=0x7f9eb806e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.048+0000 7f9ebf200640 1 --2- 172.21.15.47:0/1241728534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb806e1b0 0x7f9eb806e590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009870 tx=0x7f9eac02a2a0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.048+0000 7f9ebf200640 1 -- 172.21.15.47:0/1241728534 shutdown_connections 2024-04-24T22:39:34.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.048+0000 7f9ebf200640 1 --2- 172.21.15.47:0/1241728534 >> v2:172.21.15.113:3300/0 conn(0x7f9eb80707e0 0x7f9eb815e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.048+0000 7f9ebf200640 1 --2- 172.21.15.47:0/1241728534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb806e1b0 0x7f9eb806e590 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.048+0000 7f9ebf200640 1 -- 172.21.15.47:0/1241728534 >> 172.21.15.47:0/1241728534 conn(0x7f9eb80cb900 msgr2=0x7f9eb80cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:34.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.048+0000 7f9ebf200640 1 -- 172.21.15.47:0/1241728534 shutdown_connections 2024-04-24T22:39:34.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.048+0000 7f9ebf200640 1 -- 172.21.15.47:0/1241728534 wait complete. 2024-04-24T22:39:34.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.050+0000 7f9ebf200640 1 Processor -- start 2024-04-24T22:39:34.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.050+0000 7f9ebf200640 1 -- start start 2024-04-24T22:39:34.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebf200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb806e1b0 0x7f9eb814eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebf200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eb80707e0 0x7f9eb80712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebf200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb8070ea0 0x7f9eb8071810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebf200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb815b2c0 con 0x7f9eb80707e0 2024-04-24T22:39:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebf200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb815e8e0 con 0x7f9eb806e1b0 2024-04-24T22:39:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebf200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb8154950 con 0x7f9eb8070ea0 2024-04-24T22:39:34.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb806e1b0 0x7f9eb814eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb806e1b0 0x7f9eb814eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:46626/0 (socket says 172.21.15.47:46626) 2024-04-24T22:39:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebde00640 1 -- 172.21.15.47:0/978786655 learned_addr learned my addr 172.21.15.47:0/978786655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebe800640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb8070ea0 0x7f9eb8071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.051+0000 7f9ebd400640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eb80707e0 0x7f9eb80712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.052+0000 7f9ebde00640 1 -- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb8070ea0 msgr2=0x7f9eb8071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.052+0000 7f9ebde00640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb8070ea0 0x7f9eb8071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.052+0000 7f9ebde00640 1 -- 172.21.15.47:0/978786655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eb80707e0 msgr2=0x7f9eb80712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.052+0000 7f9ebde00640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eb80707e0 0x7f9eb80712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.052+0000 7f9ebde00640 1 -- 172.21.15.47:0/978786655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb814f960 con 0x7f9eb806e1b0 2024-04-24T22:39:34.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.052+0000 7f9ebde00640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb806e1b0 0x7f9eb814eed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009870 tx=0x7f9eac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:34.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.052+0000 7f9eb2000640 1 -- 172.21.15.47:0/978786655 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac041020 con 0x7f9eb806e1b0 2024-04-24T22:39:34.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.052+0000 7f9eb2000640 1 -- 172.21.15.47:0/978786655 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eac030640 con 0x7f9eb806e1b0 2024-04-24T22:39:34.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.052+0000 7f9eb2000640 1 -- 172.21.15.47:0/978786655 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac037040 con 0x7f9eb806e1b0 2024-04-24T22:39:34.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.053+0000 7f9ebf200640 1 -- 172.21.15.47:0/978786655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb814f410 con 0x7f9eb806e1b0 2024-04-24T22:39:34.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.053+0000 7f9ebf200640 1 -- 172.21.15.47:0/978786655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb81573d0 con 0x7f9eb806e1b0 2024-04-24T22:39:34.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.054+0000 7f9ebf200640 1 -- 172.21.15.47:0/978786655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb8068d50 con 0x7f9eb806e1b0 2024-04-24T22:39:34.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.055+0000 7f9eb2000640 1 -- 172.21.15.47:0/978786655 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f9eac044020 con 0x7f9eb806e1b0 2024-04-24T22:39:34.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.055+0000 7f9eb2000640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9e9c06bda0 0x7f9e9c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.055+0000 7f9eb2000640 1 -- 172.21.15.47:0/978786655 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f9eac0ab0f0 con 0x7f9eb806e1b0 2024-04-24T22:39:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.056+0000 7f9ebd400640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9e9c06bda0 0x7f9e9c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.056+0000 7f9ebd400640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9e9c06bda0 0x7f9e9c06e260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9eb806e700 tx=0x7f9ea8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:34.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.060+0000 7f9eb2000640 1 -- 172.21.15.47:0/978786655 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9eac04b030 con 0x7f9eb806e1b0 2024-04-24T22:39:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.270+0000 7f9ebf200640 1 -- 172.21.15.47:0/978786655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9eb804f5e0 con 0x7f9eb806e1b0 2024-04-24T22:39:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.279+0000 7f9eb2000640 1 -- 172.21.15.47:0/978786655 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f9eac079ed0 con 0x7f9eb806e1b0 2024-04-24T22:39:34.281 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:34.281 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-04-24T22:39:34.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:39:34.281 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-04-24T22:39:34.281 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-04-24T22:39:34.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:39:34.281 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-04-24T22:39:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 -- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9e9c06bda0 msgr2=0x7f9e9c06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9e9c06bda0 0x7f9e9c06e260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9eb806e700 tx=0x7f9ea8008040 comp rx=0 tx=0).stop 2024-04-24T22:39:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 -- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb806e1b0 msgr2=0x7f9eb814eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb806e1b0 0x7f9eb814eed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009870 tx=0x7f9eac0046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 -- 172.21.15.47:0/978786655 shutdown_connections 2024-04-24T22:39:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9e9c06bda0 0x7f9e9c06e260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9eb8070ea0 0x7f9eb8071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9eb80707e0 0x7f9eb80712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 --2- 172.21.15.47:0/978786655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb806e1b0 0x7f9eb814eed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 -- 172.21.15.47:0/978786655 >> 172.21.15.47:0/978786655 conn(0x7f9eb80cb900 msgr2=0x7f9eb80c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 -- 172.21.15.47:0/978786655 shutdown_connections 2024-04-24T22:39:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.282+0000 7f9ebf200640 1 -- 172.21.15.47:0/978786655 wait complete. 2024-04-24T22:39:34.305 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T22:39:34.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T22:39:34.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.524+0000 7fd719600640 1 Processor -- start 2024-04-24T22:39:34.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.524+0000 7fd719600640 1 -- start start 2024-04-24T22:39:34.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.525+0000 7fd719600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd71414e090 0x7fd7140c4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.525+0000 7fd719600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7140c3130 0x7fd7140c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.525+0000 7fd719600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7140c7dd0 con 0x7fd7140c8200 2024-04-24T22:39:34.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.525+0000 7fd719600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd7140c0f40 con 0x7fd7140c3130 2024-04-24T22:39:34.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.525+0000 7fd719600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd7140c0ca0 con 0x7fd71414e090 2024-04-24T22:39:34.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.526+0000 7fd713e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd71414e090 0x7fd7140c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.526+0000 7fd713400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7140c3130 0x7fd7140c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.526+0000 7fd718c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd7140c8200 0x7fd714157490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:54052/0 (socket says 172.21.15.47:54052) 2024-04-24T22:39:34.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.526+0000 7fd718c00640 1 -- 172.21.15.47:0/756548949 learned_addr learned my addr 172.21.15.47:0/756548949 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:34.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.526+0000 7fd713e00640 1 -- 172.21.15.47:0/756548949 >> v2:172.21.15.113:3300/0 conn(0x7fd7140c3130 msgr2=0x7fd7140c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:34.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.526+0000 7fd713e00640 1 --2- 172.21.15.47:0/756548949 >> v2:172.21.15.113:3300/0 conn(0x7fd7140c3130 0x7fd7140c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.526+0000 7fd713e00640 1 -- 172.21.15.47:0/756548949 >> v1:172.21.15.113:6789/0 conn(0x7fd7140c8200 legacy=0x7fd714157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.526+0000 7fd713e00640 1 -- 172.21.15.47:0/756548949 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7140c1340 con 0x7fd71414e090 2024-04-24T22:39:34.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.527+0000 7fd713e00640 1 --2- 172.21.15.47:0/756548949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd71414e090 0x7fd7140c4940 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd704009d90 tx=0x7fd70402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd3a3e0580cb25f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:34.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.527+0000 7fd712a00640 1 -- 172.21.15.47:0/756548949 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd704031600 con 0x7fd71414e090 2024-04-24T22:39:34.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.528+0000 7fd712a00640 1 -- 172.21.15.47:0/756548949 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7040317a0 con 0x7fd71414e090 2024-04-24T22:39:34.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.528+0000 7fd712a00640 1 -- 172.21.15.47:0/756548949 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd704031b70 con 0x7fd71414e090 2024-04-24T22:39:34.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.528+0000 7fd719600640 1 -- 172.21.15.47:0/756548949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd71414e090 msgr2=0x7fd7140c4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.528+0000 7fd719600640 1 --2- 172.21.15.47:0/756548949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd71414e090 0x7fd7140c4940 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd704009d90 tx=0x7fd70402a5d0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.528+0000 7fd719600640 1 -- 172.21.15.47:0/756548949 shutdown_connections 2024-04-24T22:39:34.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.528+0000 7fd719600640 1 --2- 172.21.15.47:0/756548949 >> v2:172.21.15.113:3300/0 conn(0x7fd7140c3130 0x7fd7140c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.528+0000 7fd719600640 1 --2- 172.21.15.47:0/756548949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd71414e090 0x7fd7140c4940 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.528+0000 7fd719600640 1 -- 172.21.15.47:0/756548949 >> 172.21.15.47:0/756548949 conn(0x7fd7140cbc30 msgr2=0x7fd7140cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:34.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.528+0000 7fd719600640 1 -- 172.21.15.47:0/756548949 shutdown_connections 2024-04-24T22:39:34.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.528+0000 7fd719600640 1 -- 172.21.15.47:0/756548949 wait complete. 2024-04-24T22:39:34.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.530+0000 7fd719600640 1 Processor -- start 2024-04-24T22:39:34.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.530+0000 7fd719600640 1 -- start start 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd719600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7140c3130 0x7fd71413f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd719600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7140c8200 0x7fd7140af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd719600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd71414e090 0x7fd7140afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd719600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd714157ba0 con 0x7fd7140c8200 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd719600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd7140c1900 con 0x7fd71414e090 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd719600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd714157d20 con 0x7fd7140c3130 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd713400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7140c8200 0x7fd7140af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd713e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7140c3130 0x7fd71413f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd713400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7140c8200 0x7fd7140af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48180/0 (socket says 172.21.15.47:48180) 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd718c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd71414e090 0x7fd7140afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.531+0000 7fd713400640 1 -- 172.21.15.47:0/1786334513 learned_addr learned my addr 172.21.15.47:0/1786334513 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.532+0000 7fd713e00640 1 -- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd71414e090 msgr2=0x7fd7140afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.532+0000 7fd713e00640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd71414e090 0x7fd7140afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.532+0000 7fd713e00640 1 -- 172.21.15.47:0/1786334513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7140c8200 msgr2=0x7fd7140af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.532+0000 7fd713e00640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7140c8200 0x7fd7140af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.532+0000 7fd713e00640 1 -- 172.21.15.47:0/1786334513 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71413e050 con 0x7fd7140c3130 2024-04-24T22:39:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.532+0000 7fd713e00640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7140c3130 0x7fd71413f420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd7140b1f70 tx=0x7fd704004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.532+0000 7fd70be00640 1 -- 172.21.15.47:0/1786334513 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70403f070 con 0x7fd7140c3130 2024-04-24T22:39:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.532+0000 7fd70be00640 1 -- 172.21.15.47:0/1786334513 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd704030c00 con 0x7fd7140c3130 2024-04-24T22:39:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.532+0000 7fd70be00640 1 -- 172.21.15.47:0/1786334513 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd704009450 con 0x7fd7140c3130 2024-04-24T22:39:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.533+0000 7fd719600640 1 -- 172.21.15.47:0/1786334513 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd714143890 con 0x7fd7140c3130 2024-04-24T22:39:34.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.533+0000 7fd719600640 1 -- 172.21.15.47:0/1786334513 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd714143e50 con 0x7fd7140c3130 2024-04-24T22:39:34.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.534+0000 7fd719600640 1 -- 172.21.15.47:0/1786334513 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd714069810 con 0x7fd7140c3130 2024-04-24T22:39:34.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.535+0000 7fd70be00640 1 -- 172.21.15.47:0/1786334513 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fd70403b410 con 0x7fd7140c3130 2024-04-24T22:39:34.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.536+0000 7fd70be00640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd6f006bd90 0x7fd6f006e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:34.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.536+0000 7fd70be00640 1 -- 172.21.15.47:0/1786334513 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd7040abae0 con 0x7fd7140c3130 2024-04-24T22:39:34.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.536+0000 7fd713400640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd6f006bd90 0x7fd6f006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:34.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.537+0000 7fd713400640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd6f006bd90 0x7fd6f006e250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd714144fe0 tx=0x7fd6fc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:34.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.542+0000 7fd70be00640 1 -- 172.21.15.47:0/1786334513 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd70407a1d0 con 0x7fd7140c3130 2024-04-24T22:39:34.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.748+0000 7fd719600640 1 -- 172.21.15.47:0/1786334513 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd71413c830 con 0x7fd7140c3130 2024-04-24T22:39:34.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:34.848+0000 7fd70be00640 1 -- 172.21.15.47:0/1786334513 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd704005d10 con 0x7fd7140c3130 2024-04-24T22:39:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.668+0000 7fd70be00640 1 -- 172.21.15.47:0/1786334513 <== mon.2 v2:172.21.15.113:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fd7040a6790 con 0x7fd7140c3130 2024-04-24T22:39:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-04-24T22:39:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.676+0000 7fd719600640 1 -- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd6f006bd90 msgr2=0x7fd6f006e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.676+0000 7fd719600640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd6f006bd90 0x7fd6f006e250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd714144fe0 tx=0x7fd6fc009190 comp rx=0 tx=0).stop 2024-04-24T22:39:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.676+0000 7fd719600640 1 -- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7140c3130 msgr2=0x7fd71413f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.676+0000 7fd719600640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7140c3130 0x7fd71413f420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd7140b1f70 tx=0x7fd704004a40 comp rx=0 tx=0).stop 2024-04-24T22:39:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.677+0000 7fd719600640 1 -- 172.21.15.47:0/1786334513 shutdown_connections 2024-04-24T22:39:35.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.677+0000 7fd719600640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd6f006bd90 0x7fd6f006e250 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.677+0000 7fd719600640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd71414e090 0x7fd7140afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.677+0000 7fd719600640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7140c8200 0x7fd7140af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.677+0000 7fd719600640 1 --2- 172.21.15.47:0/1786334513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7140c3130 0x7fd71413f420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.677+0000 7fd719600640 1 -- 172.21.15.47:0/1786334513 >> 172.21.15.47:0/1786334513 conn(0x7fd7140cbc30 msgr2=0x7fd71414c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:35.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.677+0000 7fd719600640 1 -- 172.21.15.47:0/1786334513 shutdown_connections 2024-04-24T22:39:35.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.677+0000 7fd719600640 1 -- 172.21.15.47:0/1786334513 wait complete. 2024-04-24T22:39:35.702 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T22:39:35.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.901+0000 7f5225000640 1 Processor -- start 2024-04-24T22:39:35.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.901+0000 7f5225000640 1 -- start start 2024-04-24T22:39:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.902+0000 7f5225000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f52201546d0 0x7f52200c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.903+0000 7f5225000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52200c9080 0x7f522006ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.903+0000 7f5225000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f522006f160 0x7f522015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.903+0000 7f5225000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f52200c98c0 con 0x7f522006f160 2024-04-24T22:39:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.903+0000 7f5225000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f522015a5e0 con 0x7f52201546d0 2024-04-24T22:39:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.903+0000 7f5225000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5220154ab0 con 0x7f52200c9080 2024-04-24T22:39:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.903+0000 7f521f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f52201546d0 0x7f52200c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.903+0000 7f521fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f522006f160 0x7f522015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.903+0000 7f521ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52200c9080 0x7f522006ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.903+0000 7f521f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f52201546d0 0x7f52200c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:46660/0 (socket says 172.21.15.47:46660) 2024-04-24T22:39:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.903+0000 7f521f400640 1 -- 172.21.15.47:0/2444069335 learned_addr learned my addr 172.21.15.47:0/2444069335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.904+0000 7f521f400640 1 -- 172.21.15.47:0/2444069335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52200c9080 msgr2=0x7f522006ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.904+0000 7f521f400640 1 --2- 172.21.15.47:0/2444069335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52200c9080 0x7f522006ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.904+0000 7f521f400640 1 -- 172.21.15.47:0/2444069335 >> v2:172.21.15.47:3300/0 conn(0x7f522006f160 msgr2=0x7f522015dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.904+0000 7f521f400640 1 --2- 172.21.15.47:0/2444069335 >> v2:172.21.15.47:3300/0 conn(0x7f522006f160 0x7f522015dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.904+0000 7f521f400640 1 -- 172.21.15.47:0/2444069335 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5220071320 con 0x7f52201546d0 2024-04-24T22:39:35.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.905+0000 7f521f400640 1 --2- 172.21.15.47:0/2444069335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 0x7f52200c75c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f522006dd30 tx=0x7f521002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48f9824ac5f6e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:35.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.905+0000 7f521e000640 1 -- 172.21.15.47:0/2444069335 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210037070 con 0x7f52201546d0 2024-04-24T22:39:35.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.905+0000 7f521e000640 1 -- 172.21.15.47:0/2444069335 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5210032440 con 0x7f52201546d0 2024-04-24T22:39:35.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.905+0000 7f521e000640 1 -- 172.21.15.47:0/2444069335 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210032810 con 0x7f52201546d0 2024-04-24T22:39:35.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.906+0000 7f5225000640 1 -- 172.21.15.47:0/2444069335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 msgr2=0x7f52200c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:35.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.906+0000 7f5225000640 1 --2- 172.21.15.47:0/2444069335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 0x7f52200c75c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f522006dd30 tx=0x7f521002a270 comp rx=0 tx=0).stop 2024-04-24T22:39:35.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.906+0000 7f5225000640 1 -- 172.21.15.47:0/2444069335 shutdown_connections 2024-04-24T22:39:35.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.906+0000 7f5225000640 1 --2- 172.21.15.47:0/2444069335 >> v2:172.21.15.47:3300/0 conn(0x7f522006f160 0x7f522015dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.906+0000 7f5225000640 1 --2- 172.21.15.47:0/2444069335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52200c9080 0x7f522006ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.906+0000 7f5225000640 1 --2- 172.21.15.47:0/2444069335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 0x7f52200c75c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.906+0000 7f5225000640 1 -- 172.21.15.47:0/2444069335 >> 172.21.15.47:0/2444069335 conn(0x7f522006cf20 msgr2=0x7f522006d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:35.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.906+0000 7f5225000640 1 -- 172.21.15.47:0/2444069335 shutdown_connections 2024-04-24T22:39:35.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.906+0000 7f5225000640 1 -- 172.21.15.47:0/2444069335 wait complete. 2024-04-24T22:39:35.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.908+0000 7f5225000640 1 Processor -- start 2024-04-24T22:39:35.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.909+0000 7f5225000640 1 -- start start 2024-04-24T22:39:35.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f5225000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f522006f160 0x7f52201f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f5225000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52200c9080 0x7f52200ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f5225000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 0x7f52200cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f5225000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f522015ae40 con 0x7f522006f160 2024-04-24T22:39:35.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f5225000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f522015ab60 con 0x7f52201546d0 2024-04-24T22:39:35.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f5225000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f522015b120 con 0x7f52200c9080 2024-04-24T22:39:35.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f521f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f522006f160 0x7f52201f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f521fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 0x7f52200cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f521ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52200c9080 0x7f52200ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f521fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 0x7f52200cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:46676/0 (socket says 172.21.15.47:46676) 2024-04-24T22:39:35.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.910+0000 7f521fe00640 1 -- 172.21.15.47:0/114774068 learned_addr learned my addr 172.21.15.47:0/114774068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:35.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f521fe00640 1 -- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52200c9080 msgr2=0x7f52200ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f521fe00640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52200c9080 0x7f52200ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f521fe00640 1 -- 172.21.15.47:0/114774068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f522006f160 msgr2=0x7f52201f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f521fe00640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f522006f160 0x7f52201f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f521fe00640 1 -- 172.21.15.47:0/114774068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52200cb500 con 0x7f52201546d0 2024-04-24T22:39:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f521fe00640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 0x7f52200cc880 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f521800eed0 tx=0x7f521800b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f5217400640 1 -- 172.21.15.47:0/114774068 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218018020 con 0x7f52201546d0 2024-04-24T22:39:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f5217400640 1 -- 172.21.15.47:0/114774068 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5218014400 con 0x7f52201546d0 2024-04-24T22:39:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f5217400640 1 -- 172.21.15.47:0/114774068 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52180147f0 con 0x7f52201546d0 2024-04-24T22:39:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f5225000640 1 -- 172.21.15.47:0/114774068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52200cb7e0 con 0x7f52201546d0 2024-04-24T22:39:35.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.911+0000 7f5225000640 1 -- 172.21.15.47:0/114774068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52200cbda0 con 0x7f52201546d0 2024-04-24T22:39:35.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.913+0000 7f5225000640 1 -- 172.21.15.47:0/114774068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5220068520 con 0x7f52201546d0 2024-04-24T22:39:35.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.914+0000 7f5217400640 1 -- 172.21.15.47:0/114774068 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5218005d10 con 0x7f52201546d0 2024-04-24T22:39:35.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.914+0000 7f5217400640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f51fc06be60 0x7f51fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:35.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.915+0000 7f5217400640 1 -- 172.21.15.47:0/114774068 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f521808b5f0 con 0x7f52201546d0 2024-04-24T22:39:35.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.915+0000 7f521f400640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f51fc06be60 0x7f51fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:35.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.916+0000 7f521f400640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f51fc06be60 0x7f51fc06e320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5210034000 tx=0x7f521002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:35.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:35.920+0000 7f5217400640 1 -- 172.21.15.47:0/114774068 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5218022030 con 0x7f52201546d0 2024-04-24T22:39:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.126+0000 7f5225000640 1 -- 172.21.15.47:0/114774068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f522004f5e0 con 0x7f52201546d0 2024-04-24T22:39:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.686+0000 7f5217400640 1 -- 172.21.15.47:0/114774068 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f5218059a00 con 0x7f52201546d0 2024-04-24T22:39:36.690 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-04-24T22:39:36.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.694+0000 7f5225000640 1 -- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f51fc06be60 msgr2=0x7f51fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.694+0000 7f5225000640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f51fc06be60 0x7f51fc06e320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5210034000 tx=0x7f521002e000 comp rx=0 tx=0).stop 2024-04-24T22:39:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.694+0000 7f5225000640 1 -- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 msgr2=0x7f52200cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.694+0000 7f5225000640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 0x7f52200cc880 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f521800eed0 tx=0x7f521800b480 comp rx=0 tx=0).stop 2024-04-24T22:39:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.695+0000 7f5225000640 1 -- 172.21.15.47:0/114774068 shutdown_connections 2024-04-24T22:39:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.695+0000 7f5225000640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f51fc06be60 0x7f51fc06e320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.695+0000 7f5225000640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52201546d0 0x7f52200cc880 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.695+0000 7f5225000640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52200c9080 0x7f52200ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.695+0000 7f5225000640 1 --2- 172.21.15.47:0/114774068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f522006f160 0x7f52201f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.695+0000 7f5225000640 1 -- 172.21.15.47:0/114774068 >> 172.21.15.47:0/114774068 conn(0x7f522006cf20 msgr2=0x7f52200c68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:36.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.698+0000 7f5225000640 1 -- 172.21.15.47:0/114774068 shutdown_connections 2024-04-24T22:39:36.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.698+0000 7f5225000640 1 -- 172.21.15.47:0/114774068 wait complete. 2024-04-24T22:39:36.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T22:39:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.917+0000 7f23c6e00640 1 Processor -- start 2024-04-24T22:39:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.918+0000 7f23c6e00640 1 -- start start 2024-04-24T22:39:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.919+0000 7f23c6e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c0152960 0x7f23c00c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.919+0000 7f23c6e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23c00c8fa0 0x7f23c015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.919+0000 7f23c6e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23c015bf60 0x7f23c0072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.919+0000 7f23c6e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f23c0158f20 con 0x7f23c00c8fa0 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.919+0000 7f23c6e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f23c01593b0 con 0x7f23c015bf60 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.919+0000 7f23c6e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f23c00c8130 con 0x7f23c0152960 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23bfe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c0152960 0x7f23c00c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23b7e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f23c00c8fa0 0x7f23c015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23c4e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23c015bf60 0x7f23c0072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23bfe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c0152960 0x7f23c00c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:55728/0 (socket says 172.21.15.47:55728) 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23c4e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23c015bf60 0x7f23c0072b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:40570/0 (socket says 172.21.15.47:40570) 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23bfe00640 1 -- 172.21.15.47:0/2193353118 learned_addr learned my addr 172.21.15.47:0/2193353118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23bfe00640 1 -- 172.21.15.47:0/2193353118 >> v2:172.21.15.113:3300/0 conn(0x7f23c015bf60 msgr2=0x7f23c0072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:36.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23bfe00640 1 --2- 172.21.15.47:0/2193353118 >> v2:172.21.15.113:3300/0 conn(0x7f23c015bf60 0x7f23c0072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23bfe00640 1 -- 172.21.15.47:0/2193353118 >> v2:172.21.15.47:3300/0 conn(0x7f23c00c8fa0 msgr2=0x7f23c015ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23bfe00640 1 --2- 172.21.15.47:0/2193353118 >> v2:172.21.15.47:3300/0 conn(0x7f23c00c8fa0 0x7f23c015ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23bfe00640 1 -- 172.21.15.47:0/2193353118 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c00c9400 con 0x7f23c0152960 2024-04-24T22:39:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.920+0000 7f23c4e00640 1 --2- 172.21.15.47:0/2193353118 >> v2:172.21.15.113:3300/0 conn(0x7f23c015bf60 0x7f23c0072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.921+0000 7f23bfe00640 1 --2- 172.21.15.47:0/2193353118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c0152960 0x7f23c00c7750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f23c015c3f0 tx=0x7f23b002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18cca42463267b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.922+0000 7f23bf400640 1 -- 172.21.15.47:0/2193353118 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0037070 con 0x7f23c0152960 2024-04-24T22:39:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.922+0000 7f23bf400640 1 -- 172.21.15.47:0/2193353118 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b0032440 con 0x7f23c0152960 2024-04-24T22:39:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.922+0000 7f23bf400640 1 -- 172.21.15.47:0/2193353118 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0032810 con 0x7f23c0152960 2024-04-24T22:39:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.922+0000 7f23c6e00640 1 -- 172.21.15.47:0/2193353118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c0152960 msgr2=0x7f23c00c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.922+0000 7f23c6e00640 1 --2- 172.21.15.47:0/2193353118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c0152960 0x7f23c00c7750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f23c015c3f0 tx=0x7f23b002a270 comp rx=0 tx=0).stop 2024-04-24T22:39:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.922+0000 7f23c6e00640 1 -- 172.21.15.47:0/2193353118 shutdown_connections 2024-04-24T22:39:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.922+0000 7f23c6e00640 1 --2- 172.21.15.47:0/2193353118 >> v2:172.21.15.113:3300/0 conn(0x7f23c015bf60 0x7f23c0072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.922+0000 7f23c6e00640 1 --2- 172.21.15.47:0/2193353118 >> v2:172.21.15.47:3300/0 conn(0x7f23c00c8fa0 0x7f23c015ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.922+0000 7f23c6e00640 1 --2- 172.21.15.47:0/2193353118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c0152960 0x7f23c00c7750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.922+0000 7f23c6e00640 1 -- 172.21.15.47:0/2193353118 >> 172.21.15.47:0/2193353118 conn(0x7f23c006e580 msgr2=0x7f23c006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.923+0000 7f23c6e00640 1 -- 172.21.15.47:0/2193353118 shutdown_connections 2024-04-24T22:39:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.923+0000 7f23c6e00640 1 -- 172.21.15.47:0/2193353118 wait complete. 2024-04-24T22:39:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.925+0000 7f23c6e00640 1 Processor -- start 2024-04-24T22:39:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.925+0000 7f23c6e00640 1 -- start start 2024-04-24T22:39:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23c6e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c00c8fa0 0x7f23c00cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23c6e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0152960 0x7f23c015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23c6e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c015bf60 0x7f23c00c3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23c6e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01588d0 con 0x7f23c0152960 2024-04-24T22:39:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23c6e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01585f0 con 0x7f23c00c8fa0 2024-04-24T22:39:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23c6e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f23c0158bb0 con 0x7f23c015bf60 2024-04-24T22:39:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0152960 0x7f23c015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23bfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c00c8fa0 0x7f23c00cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0152960 0x7f23c015a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53490/0 (socket says 172.21.15.47:53490) 2024-04-24T22:39:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23c4e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c015bf60 0x7f23c00c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.926+0000 7f23b7e00640 1 -- 172.21.15.47:0/3989632728 learned_addr learned my addr 172.21.15.47:0/3989632728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23bfe00640 1 -- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c015bf60 msgr2=0x7f23c00c3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23bfe00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c015bf60 0x7f23c00c3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23bfe00640 1 -- 172.21.15.47:0/3989632728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0152960 msgr2=0x7f23c015a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23bfe00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0152960 0x7f23c015a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23bfe00640 1 -- 172.21.15.47:0/3989632728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c00c4bb0 con 0x7f23c00c8fa0 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23b7e00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0152960 0x7f23c015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23c4e00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c015bf60 0x7f23c00c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23bfe00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c00c8fa0 0x7f23c00cc3a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f23b0031000 tx=0x7f23b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23bcc00640 1 -- 172.21.15.47:0/3989632728 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0041020 con 0x7f23c00c8fa0 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23bcc00640 1 -- 172.21.15.47:0/3989632728 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b003d660 con 0x7f23c00c8fa0 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23bcc00640 1 -- 172.21.15.47:0/3989632728 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0037040 con 0x7f23c00c8fa0 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.927+0000 7f23c6e00640 1 -- 172.21.15.47:0/3989632728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c01f2410 con 0x7f23c00c8fa0 2024-04-24T22:39:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.928+0000 7f23c6e00640 1 -- 172.21.15.47:0/3989632728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c00c4e50 con 0x7f23c00c8fa0 2024-04-24T22:39:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.929+0000 7f23c6e00640 1 -- 172.21.15.47:0/3989632728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c0157e40 con 0x7f23c00c8fa0 2024-04-24T22:39:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.930+0000 7f23bcc00640 1 -- 172.21.15.47:0/3989632728 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23b0044020 con 0x7f23c00c8fa0 2024-04-24T22:39:36.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.930+0000 7f23bcc00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f239c06be60 0x7f239c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:36.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.931+0000 7f23bcc00640 1 -- 172.21.15.47:0/3989632728 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f23b00ab2f0 con 0x7f23c00c8fa0 2024-04-24T22:39:36.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.931+0000 7f23b7e00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f239c06be60 0x7f239c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:36.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.931+0000 7f23b7e00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f239c06be60 0x7f239c06e320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f23c006f020 tx=0x7f23a8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:36.935+0000 7f23bcc00640 1 -- 172.21.15.47:0/3989632728 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23c0157e40 con 0x7f23c00c8fa0 2024-04-24T22:39:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.175+0000 7f23c6e00640 1 -- 172.21.15.47:0/3989632728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f23c0073b90 con 0x7f23c00c8fa0 2024-04-24T22:39:37.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.696+0000 7f23bcc00640 1 -- 172.21.15.47:0/3989632728 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f23b004b030 con 0x7f23c00c8fa0 2024-04-24T22:39:37.699 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T22:39:37.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.707+0000 7f23c6e00640 1 -- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f239c06be60 msgr2=0x7f239c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.707+0000 7f23c6e00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f239c06be60 0x7f239c06e320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f23c006f020 tx=0x7f23a8009190 comp rx=0 tx=0).stop 2024-04-24T22:39:37.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.708+0000 7f23c6e00640 1 -- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c00c8fa0 msgr2=0x7f23c00cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.708+0000 7f23c6e00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c00c8fa0 0x7f23c00cc3a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f23b0031000 tx=0x7f23b00046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.708+0000 7f23c6e00640 1 -- 172.21.15.47:0/3989632728 shutdown_connections 2024-04-24T22:39:37.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.708+0000 7f23c6e00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f239c06be60 0x7f239c06e320 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.708+0000 7f23c6e00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23c015bf60 0x7f23c00c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.708+0000 7f23c6e00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0152960 0x7f23c015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.708+0000 7f23c6e00640 1 --2- 172.21.15.47:0/3989632728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c00c8fa0 0x7f23c00cc3a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.708+0000 7f23c6e00640 1 -- 172.21.15.47:0/3989632728 >> 172.21.15.47:0/3989632728 conn(0x7f23c006e580 msgr2=0x7f23c0157180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.709+0000 7f23c6e00640 1 -- 172.21.15.47:0/3989632728 shutdown_connections 2024-04-24T22:39:37.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.709+0000 7f23c6e00640 1 -- 172.21.15.47:0/3989632728 wait complete. 2024-04-24T22:39:37.734 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:37.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.902+0000 7fe455200640 1 Processor -- start 2024-04-24T22:39:37.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.902+0000 7fe455200640 1 -- start start 2024-04-24T22:39:37.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.904+0000 7fe455200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4500c7750 0x7fe4500c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.904+0000 7fe455200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe450072b80 0x7fe450073010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.904+0000 7fe455200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4500c7bb0 con 0x7fe450152960 2024-04-24T22:39:37.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.904+0000 7fe455200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe450158f20 con 0x7fe450072b80 2024-04-24T22:39:37.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.904+0000 7fe455200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe4501593b0 con 0x7fe4500c7750 2024-04-24T22:39:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.904+0000 7fe44de00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe450152960 0x7fe45015ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:33048/0 (socket says 172.21.15.47:33048) 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.904+0000 7fe44de00640 1 -- 172.21.15.47:0/1421418528 learned_addr learned my addr 172.21.15.47:0/1421418528 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.904+0000 7fe44e800640 1 --2- 172.21.15.47:0/1421418528 >> v2:172.21.15.47:3300/0 conn(0x7fe450072b80 0x7fe450073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.904+0000 7fe44d400640 1 --2- 172.21.15.47:0/1421418528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4500c7750 0x7fe4500c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.905+0000 7fe44ca00640 1 -- 172.21.15.47:0/1421418528 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781983288 0 0) 0x7fe4500c7bb0 con 0x7fe450152960 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.905+0000 7fe44ca00640 1 -- 172.21.15.47:0/1421418528 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe434003520 con 0x7fe450152960 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.905+0000 7fe44d400640 1 -- 172.21.15.47:0/1421418528 >> v2:172.21.15.47:3300/0 conn(0x7fe450072b80 msgr2=0x7fe450073010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.905+0000 7fe44d400640 1 --2- 172.21.15.47:0/1421418528 >> v2:172.21.15.47:3300/0 conn(0x7fe450072b80 0x7fe450073010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.905+0000 7fe44d400640 1 -- 172.21.15.47:0/1421418528 >> v1:172.21.15.113:6789/0 conn(0x7fe450152960 legacy=0x7fe45015ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.905+0000 7fe44d400640 1 -- 172.21.15.47:0/1421418528 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4500c7e50 con 0x7fe4500c7750 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.905+0000 7fe44e800640 1 --2- 172.21.15.47:0/1421418528 >> v2:172.21.15.47:3300/0 conn(0x7fe450072b80 0x7fe450073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.906+0000 7fe44d400640 1 --2- 172.21.15.47:0/1421418528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4500c7750 0x7fe4500c8fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe4380097a0 tx=0x7fe43802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43dbaf278823241f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:37.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.906+0000 7fe44ca00640 1 -- 172.21.15.47:0/1421418528 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438037070 con 0x7fe4500c7750 2024-04-24T22:39:37.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.906+0000 7fe44ca00640 1 -- 172.21.15.47:0/1421418528 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe438032440 con 0x7fe4500c7750 2024-04-24T22:39:37.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.906+0000 7fe44ca00640 1 -- 172.21.15.47:0/1421418528 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4380327f0 con 0x7fe4500c7750 2024-04-24T22:39:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.907+0000 7fe455200640 1 -- 172.21.15.47:0/1421418528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4500c7750 msgr2=0x7fe4500c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.907+0000 7fe455200640 1 --2- 172.21.15.47:0/1421418528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4500c7750 0x7fe4500c8fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe4380097a0 tx=0x7fe43802a230 comp rx=0 tx=0).stop 2024-04-24T22:39:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.907+0000 7fe455200640 1 -- 172.21.15.47:0/1421418528 shutdown_connections 2024-04-24T22:39:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.907+0000 7fe455200640 1 --2- 172.21.15.47:0/1421418528 >> v2:172.21.15.47:3300/0 conn(0x7fe450072b80 0x7fe450073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.907+0000 7fe455200640 1 --2- 172.21.15.47:0/1421418528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4500c7750 0x7fe4500c8fa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.907+0000 7fe455200640 1 -- 172.21.15.47:0/1421418528 >> 172.21.15.47:0/1421418528 conn(0x7fe45006e580 msgr2=0x7fe45006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.907+0000 7fe455200640 1 -- 172.21.15.47:0/1421418528 shutdown_connections 2024-04-24T22:39:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.907+0000 7fe455200640 1 -- 172.21.15.47:0/1421418528 wait complete. 2024-04-24T22:39:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.908+0000 7fe455200640 1 Processor -- start 2024-04-24T22:39:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.909+0000 7fe455200640 1 -- start start 2024-04-24T22:39:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.909+0000 7fe455200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe450072b80 0x7fe4500c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.909+0000 7fe455200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4500c7750 0x7fe450150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.909+0000 7fe455200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe450152960 0x7fe4501f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.909+0000 7fe455200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe4501588d0 con 0x7fe4500c7750 2024-04-24T22:39:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.909+0000 7fe455200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe4500c8130 con 0x7fe450072b80 2024-04-24T22:39:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.909+0000 7fe455200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe450158bb0 con 0x7fe450152960 2024-04-24T22:39:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4500c7750 0x7fe450150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4500c7750 0x7fe450150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53520/0 (socket says 172.21.15.47:53520) 2024-04-24T22:39:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44d400640 1 -- 172.21.15.47:0/4270885323 learned_addr learned my addr 172.21.15.47:0/4270885323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44de00640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe450072b80 0x7fe4500c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44e800640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe450152960 0x7fe4501f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44d400640 1 -- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe450152960 msgr2=0x7fe4501f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44d400640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe450152960 0x7fe4501f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44d400640 1 -- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe450072b80 msgr2=0x7fe4500c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44d400640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe450072b80 0x7fe4500c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44d400640 1 -- 172.21.15.47:0/4270885323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4500c5180 con 0x7fe4500c7750 2024-04-24T22:39:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.910+0000 7fe44d400640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4500c7750 0x7fe450150d40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe43802a270 tx=0x7fe4380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.911+0000 7fe446000640 1 -- 172.21.15.47:0/4270885323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438037040 con 0x7fe4500c7750 2024-04-24T22:39:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.911+0000 7fe446000640 1 -- 172.21.15.47:0/4270885323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe438042070 con 0x7fe4500c7750 2024-04-24T22:39:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.911+0000 7fe446000640 1 -- 172.21.15.47:0/4270885323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43803b870 con 0x7fe4500c7750 2024-04-24T22:39:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.911+0000 7fe455200640 1 -- 172.21.15.47:0/4270885323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4501f03f0 con 0x7fe4500c7750 2024-04-24T22:39:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.911+0000 7fe455200640 1 -- 172.21.15.47:0/4270885323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4501f09b0 con 0x7fe4500c7750 2024-04-24T22:39:37.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.912+0000 7fe455200640 1 -- 172.21.15.47:0/4270885323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe450073b90 con 0x7fe4500c7750 2024-04-24T22:39:37.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.913+0000 7fe446000640 1 -- 172.21.15.47:0/4270885323 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe43803ba10 con 0x7fe4500c7750 2024-04-24T22:39:37.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.914+0000 7fe446000640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe42c06be30 0x7fe42c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.914+0000 7fe446000640 1 -- 172.21.15.47:0/4270885323 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe4380ab520 con 0x7fe4500c7750 2024-04-24T22:39:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.915+0000 7fe44de00640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe42c06be30 0x7fe42c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.915+0000 7fe44de00640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe42c06be30 0x7fe42c06e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe4400011e0 tx=0x7fe440007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:37.919+0000 7fe446000640 1 -- 172.21.15.47:0/4270885323 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe438079f60 con 0x7fe4500c7750 2024-04-24T22:39:38.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.123+0000 7fe455200640 1 -- 172.21.15.47:0/4270885323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe450152150 con 0x7fe4500c7750 2024-04-24T22:39:38.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.125+0000 7fe446000640 1 -- 172.21.15.47:0/4270885323 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16606 (secure 0 0 0) 0x7fe438079860 con 0x7fe4500c7750 2024-04-24T22:39:38.127 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:38.127 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:37.177407+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:38.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.127+0000 7fe455200640 1 -- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe42c06be30 msgr2=0x7fe42c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.127+0000 7fe455200640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe42c06be30 0x7fe42c06e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe4400011e0 tx=0x7fe440007d30 comp rx=0 tx=0).stop 2024-04-24T22:39:38.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.127+0000 7fe455200640 1 -- 172.21.15.47:0/4270885323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4500c7750 msgr2=0x7fe450150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.127+0000 7fe455200640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4500c7750 0x7fe450150d40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe43802a270 tx=0x7fe4380046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.127+0000 7fe455200640 1 -- 172.21.15.47:0/4270885323 shutdown_connections 2024-04-24T22:39:38.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.127+0000 7fe455200640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe42c06be30 0x7fe42c06e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.128+0000 7fe455200640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe450152960 0x7fe4501f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.128+0000 7fe455200640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4500c7750 0x7fe450150d40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.128+0000 7fe455200640 1 --2- 172.21.15.47:0/4270885323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe450072b80 0x7fe4500c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.128+0000 7fe455200640 1 -- 172.21.15.47:0/4270885323 >> 172.21.15.47:0/4270885323 conn(0x7fe45006e580 msgr2=0x7fe45006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:38.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.128+0000 7fe455200640 1 -- 172.21.15.47:0/4270885323 shutdown_connections 2024-04-24T22:39:38.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.128+0000 7fe455200640 1 -- 172.21.15.47:0/4270885323 wait complete. 2024-04-24T22:39:38.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.352+0000 7f67faa00640 1 Processor -- start 2024-04-24T22:39:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.352+0000 7f67faa00640 1 -- start start 2024-04-24T22:39:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.353+0000 7f67faa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67f4158be0 0x7f67f40c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.353+0000 7f67faa00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67f41621d0 con 0x7f67f4158fc0 2024-04-24T22:39:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.354+0000 7f67faa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67f41624b0 con 0x7f67f40c1280 2024-04-24T22:39:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.354+0000 7f67faa00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f67f4162790 con 0x7f67f4158be0 2024-04-24T22:39:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.354+0000 7f67f3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67f4158be0 0x7f67f40c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.354+0000 7f67f3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f67f4158be0 0x7f67f40c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53532/0 (socket says 172.21.15.47:53532) 2024-04-24T22:39:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.354+0000 7f67f3e00640 1 -- 172.21.15.47:0/2679323702 learned_addr learned my addr 172.21.15.47:0/2679323702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.354+0000 7f67f3400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f67f4158fc0 0x7f67f40c8590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54402/0 (socket says 172.21.15.47:54402) 2024-04-24T22:39:38.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.354+0000 7f67f3e00640 1 -- 172.21.15.47:0/2679323702 >> v1:172.21.15.113:6789/0 conn(0x7f67f40c1280 legacy=0x7f67f415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.354+0000 7f67f3e00640 1 -- 172.21.15.47:0/2679323702 >> v1:172.21.15.47:6789/0 conn(0x7f67f4158fc0 legacy=0x7f67f40c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.354+0000 7f67f2a00640 1 -- 172.21.15.47:0/2679323702 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2552016940 0 0) 0x7f67f41624b0 con 0x7f67f40c1280 2024-04-24T22:39:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.354+0000 7f67f3e00640 1 -- 172.21.15.47:0/2679323702 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f4162a70 con 0x7f67f4158be0 2024-04-24T22:39:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.356+0000 7f67f3e00640 1 --2- 172.21.15.47:0/2679323702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67f4158be0 0x7f67f40c78f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009e50 tx=0x7f67e402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15f24b3d54c2fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.356+0000 7f67f2a00640 1 -- 172.21.15.47:0/2679323702 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4036070 con 0x7f67f4158be0 2024-04-24T22:39:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.356+0000 7f67f2a00640 1 -- 172.21.15.47:0/2679323702 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e4031600 con 0x7f67f4158be0 2024-04-24T22:39:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.356+0000 7f67f2a00640 1 -- 172.21.15.47:0/2679323702 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40319d0 con 0x7f67f4158be0 2024-04-24T22:39:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.356+0000 7f67faa00640 1 -- 172.21.15.47:0/2679323702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67f4158be0 msgr2=0x7f67f40c78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.356+0000 7f67faa00640 1 --2- 172.21.15.47:0/2679323702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67f4158be0 0x7f67f40c78f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009e50 tx=0x7f67e402a750 comp rx=0 tx=0).stop 2024-04-24T22:39:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.357+0000 7f67faa00640 1 -- 172.21.15.47:0/2679323702 shutdown_connections 2024-04-24T22:39:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.357+0000 7f67faa00640 1 --2- 172.21.15.47:0/2679323702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67f4158be0 0x7f67f40c78f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.357+0000 7f67faa00640 1 -- 172.21.15.47:0/2679323702 >> 172.21.15.47:0/2679323702 conn(0x7f67f406e580 msgr2=0x7f67f406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.357+0000 7f67faa00640 1 -- 172.21.15.47:0/2679323702 shutdown_connections 2024-04-24T22:39:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.357+0000 7f67faa00640 1 -- 172.21.15.47:0/2679323702 wait complete. 2024-04-24T22:39:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.358+0000 7f67faa00640 1 Processor -- start 2024-04-24T22:39:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.358+0000 7f67faa00640 1 -- start start 2024-04-24T22:39:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.359+0000 7f67faa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f40c1280 0x7f67f414dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.359+0000 7f67faa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67f4158be0 0x7f67f415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.359+0000 7f67faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67f4158fc0 0x7f67f415fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.359+0000 7f67faa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f67f40c8e20 con 0x7f67f4158fc0 2024-04-24T22:39:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.359+0000 7f67faa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67f40c8ca0 con 0x7f67f40c1280 2024-04-24T22:39:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.359+0000 7f67faa00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f67f40c0ed0 con 0x7f67f4158be0 2024-04-24T22:39:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67f4158fc0 0x7f67f415fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67f4158be0 0x7f67f415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f40c1280 0x7f67f414dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67f4158fc0 0x7f67f415fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53548/0 (socket says 172.21.15.47:53548) 2024-04-24T22:39:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f40c1280 0x7f67f414dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:40612/0 (socket says 172.21.15.47:40612) 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f8a00640 1 -- 172.21.15.47:0/3346737244 learned_addr learned my addr 172.21.15.47:0/3346737244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f3e00640 1 -- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67f4158be0 msgr2=0x7f67f415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f3e00640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67f4158be0 0x7f67f415a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f3e00640 1 -- 172.21.15.47:0/3346737244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67f4158fc0 msgr2=0x7f67f415fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f3e00640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67f4158fc0 0x7f67f415fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.360+0000 7f67f3e00640 1 -- 172.21.15.47:0/3346737244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f41f5340 con 0x7f67f40c1280 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.361+0000 7f67f3e00640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f40c1280 0x7f67f414dd00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67e4030000 tx=0x7f67e4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.361+0000 7f67ebe00640 1 -- 172.21.15.47:0/3346737244 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4041070 con 0x7f67f40c1280 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.361+0000 7f67ebe00640 1 -- 172.21.15.47:0/3346737244 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e4003990 con 0x7f67f40c1280 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.361+0000 7f67faa00640 1 -- 172.21.15.47:0/3346737244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f415e470 con 0x7f67f40c1280 2024-04-24T22:39:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.361+0000 7f67ebe00640 1 -- 172.21.15.47:0/3346737244 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4036040 con 0x7f67f40c1280 2024-04-24T22:39:38.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.361+0000 7f67faa00640 1 -- 172.21.15.47:0/3346737244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f415e710 con 0x7f67f40c1280 2024-04-24T22:39:38.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.362+0000 7f67faa00640 1 -- 172.21.15.47:0/3346737244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f4073b90 con 0x7f67f40c1280 2024-04-24T22:39:38.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.363+0000 7f67ebe00640 1 -- 172.21.15.47:0/3346737244 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67e4007820 con 0x7f67f40c1280 2024-04-24T22:39:38.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.364+0000 7f67ebe00640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f67d006be30 0x7f67d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.364+0000 7f67ebe00640 1 -- 172.21.15.47:0/3346737244 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f67e40aba70 con 0x7f67f40c1280 2024-04-24T22:39:38.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.364+0000 7f67f3400640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f67d006be30 0x7f67d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.365+0000 7f67f3400640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f67d006be30 0x7f67d006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67f4151e60 tx=0x7f67dc007e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:38.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.369+0000 7f67ebe00640 1 -- 172.21.15.47:0/3346737244 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f67e4079dc0 con 0x7f67f40c1280 2024-04-24T22:39:38.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.571+0000 7f67faa00640 1 -- 172.21.15.47:0/3346737244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67f40c76d0 con 0x7f67f40c1280 2024-04-24T22:39:38.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.573+0000 7f67ebe00640 1 -- 172.21.15.47:0/3346737244 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16606 (secure 0 0 0) 0x7f67e407e0c0 con 0x7f67f40c1280 2024-04-24T22:39:38.575 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:38.575 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:37.177407+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.575+0000 7f67faa00640 1 -- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f67d006be30 msgr2=0x7f67d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.575+0000 7f67faa00640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f67d006be30 0x7f67d006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67f4151e60 tx=0x7f67dc007e20 comp rx=0 tx=0).stop 2024-04-24T22:39:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.575+0000 7f67faa00640 1 -- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f40c1280 msgr2=0x7f67f414dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.575+0000 7f67faa00640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f40c1280 0x7f67f414dd00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67e4030000 tx=0x7f67e4004a40 comp rx=0 tx=0).stop 2024-04-24T22:39:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.575+0000 7f67faa00640 1 -- 172.21.15.47:0/3346737244 shutdown_connections 2024-04-24T22:39:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.575+0000 7f67faa00640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f67d006be30 0x7f67d006e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.575+0000 7f67faa00640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67f4158fc0 0x7f67f415fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.575+0000 7f67faa00640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67f4158be0 0x7f67f415a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.575+0000 7f67faa00640 1 --2- 172.21.15.47:0/3346737244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67f40c1280 0x7f67f414dd00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.575+0000 7f67faa00640 1 -- 172.21.15.47:0/3346737244 >> 172.21.15.47:0/3346737244 conn(0x7f67f406e580 msgr2=0x7f67f4071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.576+0000 7f67faa00640 1 -- 172.21.15.47:0/3346737244 shutdown_connections 2024-04-24T22:39:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.576+0000 7f67faa00640 1 -- 172.21.15.47:0/3346737244 wait complete. 2024-04-24T22:39:38.597 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T22:39:38.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.805+0000 7fe347200640 1 Processor -- start 2024-04-24T22:39:38.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.805+0000 7fe347200640 1 -- start start 2024-04-24T22:39:38.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.807+0000 7fe347200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe34806e1b0 0x7fe34806e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.807+0000 7fe347200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe348070ea0 0x7fe348164d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.807+0000 7fe347200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3480c6f30 con 0x7fe3480707e0 2024-04-24T22:39:38.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.807+0000 7fe347200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe3480c7210 con 0x7fe348070ea0 2024-04-24T22:39:38.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.807+0000 7fe347200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe34815a500 con 0x7fe34806e1b0 2024-04-24T22:39:38.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.807+0000 7fe345400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe3480707e0 0x7fe34815e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54408/0 (socket says 172.21.15.47:54408) 2024-04-24T22:39:38.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.807+0000 7fe345400640 1 -- 172.21.15.47:0/414929166 learned_addr learned my addr 172.21.15.47:0/414929166 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:38.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.807+0000 7fe346800640 1 --2- 172.21.15.47:0/414929166 >> v2:172.21.15.47:3300/0 conn(0x7fe348070ea0 0x7fe348164d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.807+0000 7fe345e00640 1 --2- 172.21.15.47:0/414929166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe34806e1b0 0x7fe34806e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.808+0000 7fe346800640 1 -- 172.21.15.47:0/414929166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe34806e1b0 msgr2=0x7fe34806e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:38.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.808+0000 7fe346800640 1 --2- 172.21.15.47:0/414929166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe34806e1b0 0x7fe34806e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.808+0000 7fe346800640 1 -- 172.21.15.47:0/414929166 >> v1:172.21.15.47:6789/0 conn(0x7fe3480707e0 legacy=0x7fe34815e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.808+0000 7fe346800640 1 -- 172.21.15.47:0/414929166 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34815a7e0 con 0x7fe348070ea0 2024-04-24T22:39:38.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.808+0000 7fe344a00640 1 -- 172.21.15.47:0/414929166 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1503809937 0 0) 0x7fe3480c6f30 con 0x7fe3480707e0 2024-04-24T22:39:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.809+0000 7fe346800640 1 --2- 172.21.15.47:0/414929166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348070ea0 0x7fe348164d70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe340009d90 tx=0x7fe34002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7d1edd62f902a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:38.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.809+0000 7fe344a00640 1 -- 172.21.15.47:0/414929166 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340007740 con 0x7fe348070ea0 2024-04-24T22:39:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.809+0000 7fe344a00640 1 -- 172.21.15.47:0/414929166 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3400078e0 con 0x7fe348070ea0 2024-04-24T22:39:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.809+0000 7fe344a00640 1 -- 172.21.15.47:0/414929166 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340007cb0 con 0x7fe348070ea0 2024-04-24T22:39:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.810+0000 7fe347200640 1 -- 172.21.15.47:0/414929166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348070ea0 msgr2=0x7fe348164d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.810+0000 7fe347200640 1 --2- 172.21.15.47:0/414929166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348070ea0 0x7fe348164d70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe340009d90 tx=0x7fe34002a6f0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.810+0000 7fe347200640 1 -- 172.21.15.47:0/414929166 shutdown_connections 2024-04-24T22:39:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.810+0000 7fe347200640 1 --2- 172.21.15.47:0/414929166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348070ea0 0x7fe348164d70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.810+0000 7fe347200640 1 --2- 172.21.15.47:0/414929166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe34806e1b0 0x7fe34806e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.810+0000 7fe347200640 1 -- 172.21.15.47:0/414929166 >> 172.21.15.47:0/414929166 conn(0x7fe3480cb900 msgr2=0x7fe3480cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.810+0000 7fe347200640 1 -- 172.21.15.47:0/414929166 shutdown_connections 2024-04-24T22:39:38.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.810+0000 7fe347200640 1 -- 172.21.15.47:0/414929166 wait complete. 2024-04-24T22:39:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.812+0000 7fe347200640 1 Processor -- start 2024-04-24T22:39:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.812+0000 7fe347200640 1 -- start start 2024-04-24T22:39:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe347200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe34806e1b0 0x7fe348151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe347200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3480707e0 0x7fe348152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe347200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe348070ea0 0x7fe3481f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe347200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe348154950 con 0x7fe3480707e0 2024-04-24T22:39:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe347200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe34815b2c0 con 0x7fe348070ea0 2024-04-24T22:39:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe347200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe348154ad0 con 0x7fe34806e1b0 2024-04-24T22:39:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe345400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3480707e0 0x7fe348152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe345400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3480707e0 0x7fe348152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53558/0 (socket says 172.21.15.47:53558) 2024-04-24T22:39:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe345400640 1 -- 172.21.15.47:0/779748153 learned_addr learned my addr 172.21.15.47:0/779748153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe346800640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe348070ea0 0x7fe3481f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe345e00640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe34806e1b0 0x7fe348151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe345400640 1 -- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe34806e1b0 msgr2=0x7fe348151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe345400640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe34806e1b0 0x7fe348151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe345400640 1 -- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe348070ea0 msgr2=0x7fe3481f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe345400640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe348070ea0 0x7fe3481f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.813+0000 7fe345400640 1 -- 172.21.15.47:0/779748153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34814c9c0 con 0x7fe3480707e0 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.814+0000 7fe345400640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3480707e0 0x7fe348152e60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c500 tx=0x7fe33400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.814+0000 7fe33e000640 1 -- 172.21.15.47:0/779748153 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334018070 con 0x7fe3480707e0 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.814+0000 7fe33e000640 1 -- 172.21.15.47:0/779748153 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe334007580 con 0x7fe3480707e0 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.814+0000 7fe33e000640 1 -- 172.21.15.47:0/779748153 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334004a20 con 0x7fe3480707e0 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.814+0000 7fe347200640 1 -- 172.21.15.47:0/779748153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3480c3bb0 con 0x7fe3480707e0 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.814+0000 7fe347200640 1 -- 172.21.15.47:0/779748153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3480c4170 con 0x7fe3480707e0 2024-04-24T22:39:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.815+0000 7fe347200640 1 -- 172.21.15.47:0/779748153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe348068d50 con 0x7fe3480707e0 2024-04-24T22:39:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.817+0000 7fe33e000640 1 -- 172.21.15.47:0/779748153 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe334004bc0 con 0x7fe3480707e0 2024-04-24T22:39:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.818+0000 7fe33e000640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe32806be30 0x7fe32806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.818+0000 7fe33e000640 1 -- 172.21.15.47:0/779748153 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe334089fe0 con 0x7fe3480707e0 2024-04-24T22:39:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.819+0000 7fe345e00640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe32806be30 0x7fe32806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.820+0000 7fe345e00640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe32806be30 0x7fe32806e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe338009aa0 tx=0x7fe338009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:38.824+0000 7fe33e000640 1 -- 172.21.15.47:0/779748153 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe34814c9c0 con 0x7fe3480707e0 2024-04-24T22:39:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.073+0000 7fe347200640 1 -- 172.21.15.47:0/779748153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe34806ea20 con 0x7fe3480707e0 2024-04-24T22:39:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.702+0000 7fe33e000640 1 -- 172.21.15.47:0/779748153 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7fe334058350 con 0x7fe3480707e0 2024-04-24T22:39:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 -- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe32806be30 msgr2=0x7fe32806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe32806be30 0x7fe32806e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe338009aa0 tx=0x7fe338009190 comp rx=0 tx=0).stop 2024-04-24T22:39:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 -- 172.21.15.47:0/779748153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3480707e0 msgr2=0x7fe348152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3480707e0 0x7fe348152e60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c500 tx=0x7fe33400c9c0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 -- 172.21.15.47:0/779748153 shutdown_connections 2024-04-24T22:39:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe32806be30 0x7fe32806e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe348070ea0 0x7fe3481f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3480707e0 0x7fe348152e60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 --2- 172.21.15.47:0/779748153 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe34806e1b0 0x7fe348151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 -- 172.21.15.47:0/779748153 >> 172.21.15.47:0/779748153 conn(0x7fe3480cb900 msgr2=0x7fe3480ca570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 -- 172.21.15.47:0/779748153 shutdown_connections 2024-04-24T22:39:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.706+0000 7fe347200640 1 -- 172.21.15.47:0/779748153 wait complete. 2024-04-24T22:39:39.727 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300} 2024-04-24T22:39:39.728 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-24T22:39:39.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:39:39.824 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:39:39.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.948+0000 7f3ae3c00640 1 Processor -- start 2024-04-24T22:39:39.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.948+0000 7f3ae3c00640 1 -- start start 2024-04-24T22:39:39.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.949+0000 7f3ae3c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3adc15a680 0x7f3adc0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.949+0000 7f3ae3c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc0c8ad0 0x7f3adc0c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.949+0000 7f3ae3c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3adc15ab10 con 0x7f3adc154960 2024-04-24T22:39:39.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.949+0000 7f3ae3c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3adc15adf0 con 0x7f3adc15a680 2024-04-24T22:39:39.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.949+0000 7f3ae3c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc15b0d0 con 0x7f3adc0c8ad0 2024-04-24T22:39:39.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.950+0000 7f3ae1200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3adc154960 0x7f3adc15e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54412/0 (socket says 172.21.15.47:54412) 2024-04-24T22:39:39.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.950+0000 7f3ae1200640 1 -- 172.21.15.47:0/2819749316 learned_addr learned my addr 172.21.15.47:0/2819749316 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:39.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.950+0000 7f3ae1c00640 1 --2- 172.21.15.47:0/2819749316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc0c8ad0 0x7f3adc0c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.950+0000 7f3adbe00640 1 --2- 172.21.15.47:0/2819749316 >> v2:172.21.15.113:3300/0 conn(0x7f3adc15a680 0x7f3adc0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.950+0000 7f3adb400640 1 -- 172.21.15.47:0/2819749316 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010182987 0 0) 0x7f3adc15ab10 con 0x7f3adc154960 2024-04-24T22:39:39.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.950+0000 7f3adb400640 1 -- 172.21.15.47:0/2819749316 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ac4003520 con 0x7f3adc154960 2024-04-24T22:39:39.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.950+0000 7f3ae1c00640 1 -- 172.21.15.47:0/2819749316 >> v2:172.21.15.113:3300/0 conn(0x7f3adc15a680 msgr2=0x7f3adc0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:39.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.950+0000 7f3ae1c00640 1 --2- 172.21.15.47:0/2819749316 >> v2:172.21.15.113:3300/0 conn(0x7f3adc15a680 0x7f3adc0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.950+0000 7f3ae1c00640 1 -- 172.21.15.47:0/2819749316 >> v1:172.21.15.47:6789/0 conn(0x7f3adc154960 legacy=0x7f3adc15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.950+0000 7f3ae1c00640 1 -- 172.21.15.47:0/2819749316 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc15b3b0 con 0x7f3adc0c8ad0 2024-04-24T22:39:39.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.951+0000 7f3ae1c00640 1 --2- 172.21.15.47:0/2819749316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc0c8ad0 0x7f3adc0c1070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009d90 tx=0x7f3ad402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0635a7454058be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:39.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.952+0000 7f3adb400640 1 -- 172.21.15.47:0/2819749316 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007740 con 0x7f3adc0c8ad0 2024-04-24T22:39:39.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.952+0000 7f3adb400640 1 -- 172.21.15.47:0/2819749316 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad40078e0 con 0x7f3adc0c8ad0 2024-04-24T22:39:39.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.952+0000 7f3adb400640 1 -- 172.21.15.47:0/2819749316 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007cb0 con 0x7f3adc0c8ad0 2024-04-24T22:39:39.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.952+0000 7f3ae3c00640 1 -- 172.21.15.47:0/2819749316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc0c8ad0 msgr2=0x7f3adc0c1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.952+0000 7f3ae3c00640 1 --2- 172.21.15.47:0/2819749316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc0c8ad0 0x7f3adc0c1070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009d90 tx=0x7f3ad402a660 comp rx=0 tx=0).stop 2024-04-24T22:39:39.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.952+0000 7f3ae3c00640 1 -- 172.21.15.47:0/2819749316 shutdown_connections 2024-04-24T22:39:39.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.952+0000 7f3ae3c00640 1 --2- 172.21.15.47:0/2819749316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc0c8ad0 0x7f3adc0c1070 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.952+0000 7f3ae3c00640 1 --2- 172.21.15.47:0/2819749316 >> v2:172.21.15.113:3300/0 conn(0x7f3adc15a680 0x7f3adc0c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.952+0000 7f3ae3c00640 1 -- 172.21.15.47:0/2819749316 >> 172.21.15.47:0/2819749316 conn(0x7f3adc06e580 msgr2=0x7f3adc06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:39.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.953+0000 7f3ae3c00640 1 -- 172.21.15.47:0/2819749316 shutdown_connections 2024-04-24T22:39:39.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.953+0000 7f3ae3c00640 1 -- 172.21.15.47:0/2819749316 wait complete. 2024-04-24T22:39:39.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.954+0000 7f3ae3c00640 1 Processor -- start 2024-04-24T22:39:39.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.954+0000 7f3ae3c00640 1 -- start start 2024-04-24T22:39:39.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.955+0000 7f3ae3c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3adc0c8ad0 0x7f3adc157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.955+0000 7f3ae3c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3adc154960 0x7f3adc143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.955+0000 7f3ae3c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc155af0 0x7f3adc159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.955+0000 7f3ae3c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc15ea00 con 0x7f3adc154960 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.955+0000 7f3ae3c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc154d40 con 0x7f3adc0c8ad0 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.955+0000 7f3ae3c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc15eb80 con 0x7f3adc155af0 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3adbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3adc154960 0x7f3adc143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3ae1200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3adc0c8ad0 0x7f3adc157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3adbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3adc154960 0x7f3adc143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53562/0 (socket says 172.21.15.47:53562) 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3adbe00640 1 -- 172.21.15.47:0/4014401438 learned_addr learned my addr 172.21.15.47:0/4014401438 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3ae1c00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc155af0 0x7f3adc159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3adbe00640 1 -- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc155af0 msgr2=0x7f3adc159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3adbe00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc155af0 0x7f3adc159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3adbe00640 1 -- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3adc0c8ad0 msgr2=0x7f3adc157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3adbe00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3adc0c8ad0 0x7f3adc157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:39.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3adbe00640 1 -- 172.21.15.47:0/4014401438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc15b8a0 con 0x7f3adc154960 2024-04-24T22:39:39.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3ae1200640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3adc0c8ad0 0x7f3adc157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:39.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.956+0000 7f3adbe00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3adc154960 0x7f3adc143f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800e870 tx=0x7f3ac800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:39.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.957+0000 7f3ad8c00640 1 -- 172.21.15.47:0/4014401438 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8017070 con 0x7f3adc154960 2024-04-24T22:39:39.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.957+0000 7f3ad8c00640 1 -- 172.21.15.47:0/4014401438 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac800c940 con 0x7f3adc154960 2024-04-24T22:39:39.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.957+0000 7f3ad8c00640 1 -- 172.21.15.47:0/4014401438 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800cd30 con 0x7f3adc154960 2024-04-24T22:39:39.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.957+0000 7f3ae3c00640 1 -- 172.21.15.47:0/4014401438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3adc15bdf0 con 0x7f3adc154960 2024-04-24T22:39:39.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.957+0000 7f3ae3c00640 1 -- 172.21.15.47:0/4014401438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3adc15c780 con 0x7f3adc154960 2024-04-24T22:39:39.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.958+0000 7f3ae3c00640 1 -- 172.21.15.47:0/4014401438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3adc073b90 con 0x7f3adc154960 2024-04-24T22:39:39.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.959+0000 7f3ad8c00640 1 -- 172.21.15.47:0/4014401438 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ac801a070 con 0x7f3adc154960 2024-04-24T22:39:39.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.960+0000 7f3ad8c00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3abc06be30 0x7f3abc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:39.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.960+0000 7f3ad8c00640 1 -- 172.21.15.47:0/4014401438 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3ac808b0c0 con 0x7f3adc154960 2024-04-24T22:39:39.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.960+0000 7f3ae1200640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3abc06be30 0x7f3abc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:39.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.961+0000 7f3ae1200640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3abc06be30 0x7f3abc06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005880 tx=0x7f3ad0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:39.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:39.965+0000 7f3ad8c00640 1 -- 172.21.15.47:0/4014401438 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ac8059380 con 0x7f3adc154960 2024-04-24T22:39:40.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.209+0000 7f3ae3c00640 1 -- 172.21.15.47:0/4014401438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f3adc148400 con 0x7f3adc154960 2024-04-24T22:39:40.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.710+0000 7f3ad8c00640 1 -- 172.21.15.47:0/4014401438 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7f3ac805d680 con 0x7f3adc154960 2024-04-24T22:39:40.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.713+0000 7f3ae3c00640 1 -- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3abc06be30 msgr2=0x7f3abc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:40.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.713+0000 7f3ae3c00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3abc06be30 0x7f3abc06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005880 tx=0x7f3ad0009040 comp rx=0 tx=0).stop 2024-04-24T22:39:40.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.713+0000 7f3ae3c00640 1 -- 172.21.15.47:0/4014401438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3adc154960 msgr2=0x7f3adc143f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.713+0000 7f3ae3c00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3adc154960 0x7f3adc143f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800e870 tx=0x7f3ac800ed30 comp rx=0 tx=0).stop 2024-04-24T22:39:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.714+0000 7f3ae3c00640 1 -- 172.21.15.47:0/4014401438 shutdown_connections 2024-04-24T22:39:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.714+0000 7f3ae3c00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3abc06be30 0x7f3abc06e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.714+0000 7f3ae3c00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3adc155af0 0x7f3adc159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.714+0000 7f3ae3c00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3adc154960 0x7f3adc143f50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.714+0000 7f3ae3c00640 1 --2- 172.21.15.47:0/4014401438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3adc0c8ad0 0x7f3adc157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.714+0000 7f3ae3c00640 1 -- 172.21.15.47:0/4014401438 >> 172.21.15.47:0/4014401438 conn(0x7f3adc06e580 msgr2=0x7f3adc150680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.714+0000 7f3ae3c00640 1 -- 172.21.15.47:0/4014401438 shutdown_connections 2024-04-24T22:39:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.714+0000 7f3ae3c00640 1 -- 172.21.15.47:0/4014401438 wait complete. 2024-04-24T22:39:40.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T22:39:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.993+0000 7f5407200640 1 Processor -- start 2024-04-24T22:39:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.994+0000 7f5407200640 1 -- start start 2024-04-24T22:39:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.995+0000 7f5407200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408072c60 0x7f54080c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.995+0000 7f5407200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f540806bb90 0x7f540815b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.995+0000 7f5407200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f540815c3d0 con 0x7f5408073040 2024-04-24T22:39:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.995+0000 7f5407200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5408158bf0 con 0x7f540806bb90 2024-04-24T22:39:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.995+0000 7f5407200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f540815c6b0 con 0x7f5408072c60 2024-04-24T22:39:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.996+0000 7f53ffe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5408073040 0x7f54081411f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:33062/0 (socket says 172.21.15.47:33062) 2024-04-24T22:39:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.996+0000 7f53ffe00640 1 -- 172.21.15.47:0/2557206557 learned_addr learned my addr 172.21.15.47:0/2557206557 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.996+0000 7f5405e00640 1 --2- 172.21.15.47:0/2557206557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408072c60 0x7f54080c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.996+0000 7f5406800640 1 --2- 172.21.15.47:0/2557206557 >> v2:172.21.15.47:3300/0 conn(0x7f540806bb90 0x7f540815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.996+0000 7f5405400640 1 -- 172.21.15.47:0/2557206557 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2958400051 0 0) 0x7f540815c3d0 con 0x7f5408073040 2024-04-24T22:39:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.996+0000 7f5405400640 1 -- 172.21.15.47:0/2557206557 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53ec003520 con 0x7f5408073040 2024-04-24T22:39:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.996+0000 7f5405e00640 1 -- 172.21.15.47:0/2557206557 >> v2:172.21.15.47:3300/0 conn(0x7f540806bb90 msgr2=0x7f540815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.996+0000 7f5405e00640 1 --2- 172.21.15.47:0/2557206557 >> v2:172.21.15.47:3300/0 conn(0x7f540806bb90 0x7f540815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.996+0000 7f5405e00640 1 -- 172.21.15.47:0/2557206557 >> v1:172.21.15.113:6789/0 conn(0x7f5408073040 legacy=0x7f54081411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.996+0000 7f5405e00640 1 -- 172.21.15.47:0/2557206557 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54081585a0 con 0x7f5408072c60 2024-04-24T22:39:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.997+0000 7f5405e00640 1 --2- 172.21.15.47:0/2557206557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408072c60 0x7f54080c2d60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f53f00097a0 tx=0x7f53f002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e2167458d9de64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.998+0000 7f5405400640 1 -- 172.21.15.47:0/2557206557 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0037070 con 0x7f5408072c60 2024-04-24T22:39:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.998+0000 7f5405400640 1 -- 172.21.15.47:0/2557206557 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f0032440 con 0x7f5408072c60 2024-04-24T22:39:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.998+0000 7f5405400640 1 -- 172.21.15.47:0/2557206557 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0032810 con 0x7f5408072c60 2024-04-24T22:39:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.998+0000 7f5407200640 1 -- 172.21.15.47:0/2557206557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408072c60 msgr2=0x7f54080c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.998+0000 7f5407200640 1 --2- 172.21.15.47:0/2557206557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408072c60 0x7f54080c2d60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f53f00097a0 tx=0x7f53f002a210 comp rx=0 tx=0).stop 2024-04-24T22:39:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.998+0000 7f5407200640 1 -- 172.21.15.47:0/2557206557 shutdown_connections 2024-04-24T22:39:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.998+0000 7f5407200640 1 --2- 172.21.15.47:0/2557206557 >> v2:172.21.15.47:3300/0 conn(0x7f540806bb90 0x7f540815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.998+0000 7f5407200640 1 --2- 172.21.15.47:0/2557206557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408072c60 0x7f54080c2d60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.998+0000 7f5407200640 1 -- 172.21.15.47:0/2557206557 >> 172.21.15.47:0/2557206557 conn(0x7f54080cb900 msgr2=0x7f54080cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.998+0000 7f5407200640 1 -- 172.21.15.47:0/2557206557 shutdown_connections 2024-04-24T22:39:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:40.999+0000 7f5407200640 1 -- 172.21.15.47:0/2557206557 wait complete. 2024-04-24T22:39:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.000+0000 7f5407200640 1 Processor -- start 2024-04-24T22:39:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.000+0000 7f5407200640 1 -- start start 2024-04-24T22:39:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.001+0000 7f5407200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540806bb90 0x7f54081ef250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.001+0000 7f5407200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5408072c60 0x7f5408154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.001+0000 7f5407200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408073040 0x7f54080b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.001+0000 7f5407200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f54080c32a0 con 0x7f5408072c60 2024-04-24T22:39:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.001+0000 7f5407200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5408158270 con 0x7f540806bb90 2024-04-24T22:39:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.001+0000 7f5407200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5408159080 con 0x7f5408073040 2024-04-24T22:39:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.001+0000 7f5405e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540806bb90 0x7f54081ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.001+0000 7f53ffe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5408072c60 0x7f5408154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:41.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.001+0000 7f5405e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540806bb90 0x7f54081ef250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:40648/0 (socket says 172.21.15.47:40648) 2024-04-24T22:39:41.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.001+0000 7f5405e00640 1 -- 172.21.15.47:0/4005450817 learned_addr learned my addr 172.21.15.47:0/4005450817 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:41.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.002+0000 7f5406800640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408073040 0x7f54080b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:41.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.002+0000 7f5405e00640 1 -- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408073040 msgr2=0x7f54080b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:41.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.002+0000 7f5405e00640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408073040 0x7f54080b8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.002+0000 7f5405e00640 1 -- 172.21.15.47:0/4005450817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5408072c60 msgr2=0x7f5408154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:41.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.002+0000 7f5405e00640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5408072c60 0x7f5408154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.002+0000 7f5405e00640 1 -- 172.21.15.47:0/4005450817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54080c8fa0 con 0x7f540806bb90 2024-04-24T22:39:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.002+0000 7f5405e00640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540806bb90 0x7f54081ef250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53f0035040 tx=0x7f53f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.002+0000 7f53fe000640 1 -- 172.21.15.47:0/4005450817 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0041030 con 0x7f540806bb90 2024-04-24T22:39:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.003+0000 7f53fe000640 1 -- 172.21.15.47:0/4005450817 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f003d4e0 con 0x7f540806bb90 2024-04-24T22:39:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.003+0000 7f53fe000640 1 -- 172.21.15.47:0/4005450817 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0037040 con 0x7f540806bb90 2024-04-24T22:39:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.003+0000 7f5407200640 1 -- 172.21.15.47:0/4005450817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54081edde0 con 0x7f540806bb90 2024-04-24T22:39:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.003+0000 7f5407200640 1 -- 172.21.15.47:0/4005450817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54081ee080 con 0x7f540806bb90 2024-04-24T22:39:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.004+0000 7f5407200640 1 -- 172.21.15.47:0/4005450817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5408068d50 con 0x7f540806bb90 2024-04-24T22:39:41.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.005+0000 7f53fe000640 1 -- 172.21.15.47:0/4005450817 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53f0044070 con 0x7f540806bb90 2024-04-24T22:39:41.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.006+0000 7f53fe000640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f53e406be60 0x7f53e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:41.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.006+0000 7f53fe000640 1 -- 172.21.15.47:0/4005450817 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f53f00ab380 con 0x7f540806bb90 2024-04-24T22:39:41.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.006+0000 7f53ffe00640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f53e406be60 0x7f53e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.007+0000 7f53ffe00640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f53e406be60 0x7f53e406e320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f54081579d0 tx=0x7f53f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:41.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.011+0000 7f53fe000640 1 -- 172.21.15.47:0/4005450817 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f53f0079650 con 0x7f540806bb90 2024-04-24T22:39:41.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.252+0000 7f5407200640 1 -- 172.21.15.47:0/4005450817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f540804f5e0 con 0x7f540806bb90 2024-04-24T22:39:41.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.715+0000 7f53fe000640 1 -- 172.21.15.47:0/4005450817 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f53f007d950 con 0x7f540806bb90 2024-04-24T22:39:41.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.719+0000 7f5407200640 1 -- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f53e406be60 msgr2=0x7f53e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.719+0000 7f5407200640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f53e406be60 0x7f53e406e320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f54081579d0 tx=0x7f53f8009040 comp rx=0 tx=0).stop 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.719+0000 7f5407200640 1 -- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540806bb90 msgr2=0x7f54081ef250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.719+0000 7f5407200640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540806bb90 0x7f54081ef250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53f0035040 tx=0x7f53f00046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.720+0000 7f5407200640 1 -- 172.21.15.47:0/4005450817 shutdown_connections 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.720+0000 7f5407200640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f53e406be60 0x7f53e406e320 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.720+0000 7f5407200640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5408073040 0x7f54080b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.720+0000 7f5407200640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5408072c60 0x7f5408154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.720+0000 7f5407200640 1 --2- 172.21.15.47:0/4005450817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540806bb90 0x7f54081ef250 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.720+0000 7f5407200640 1 -- 172.21.15.47:0/4005450817 >> 172.21.15.47:0/4005450817 conn(0x7f54080cb900 msgr2=0x7f540813fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.720+0000 7f5407200640 1 -- 172.21.15.47:0/4005450817 shutdown_connections 2024-04-24T22:39:41.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.720+0000 7f5407200640 1 -- 172.21.15.47:0/4005450817 wait complete. 2024-04-24T22:39:41.742 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-24T22:39:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.952+0000 7fb8abe00640 1 Processor -- start 2024-04-24T22:39:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.952+0000 7fb8abe00640 1 -- start start 2024-04-24T22:39:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.953+0000 7fb8abe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8ac072c60 0x7fb8ac0c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.953+0000 7fb8abe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8ac06bb90 0x7fb8ac15b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.953+0000 7fb8abe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8ac15c3d0 con 0x7fb8ac073040 2024-04-24T22:39:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.953+0000 7fb8abe00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb8ac158bf0 con 0x7fb8ac072c60 2024-04-24T22:39:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.953+0000 7fb8abe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb8ac15c6b0 con 0x7fb8ac06bb90 2024-04-24T22:39:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.954+0000 7fb8ab400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8ac06bb90 0x7fb8ac15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.954+0000 7fb8aaa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb8ac072c60 0x7fb8ac0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.954+0000 7fb8aa000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb8ac073040 0x7fb8ac1411f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:33068/0 (socket says 172.21.15.47:33068) 2024-04-24T22:39:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.954+0000 7fb8aa000640 1 -- 172.21.15.47:0/3982329468 learned_addr learned my addr 172.21.15.47:0/3982329468 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.954+0000 7fb8ab400640 1 -- 172.21.15.47:0/3982329468 >> v2:172.21.15.47:3300/0 conn(0x7fb8ac072c60 msgr2=0x7fb8ac0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.954+0000 7fb8ab400640 1 --2- 172.21.15.47:0/3982329468 >> v2:172.21.15.47:3300/0 conn(0x7fb8ac072c60 0x7fb8ac0c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.954+0000 7fb8ab400640 1 -- 172.21.15.47:0/3982329468 >> v1:172.21.15.113:6789/0 conn(0x7fb8ac073040 legacy=0x7fb8ac1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.954+0000 7fb8ab400640 1 -- 172.21.15.47:0/3982329468 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac1585a0 con 0x7fb8ac06bb90 2024-04-24T22:39:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.954+0000 7fb8a9600640 1 -- 172.21.15.47:0/3982329468 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2583893325 0 0) 0x7fb8ac15c3d0 con 0x7fb8ac073040 2024-04-24T22:39:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.955+0000 7fb8ab400640 1 --2- 172.21.15.47:0/3982329468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8ac06bb90 0x7fb8ac15b860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac158b60 tx=0x7fb8a402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f13f1a0306a6205 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8a9600640 1 -- 172.21.15.47:0/3982329468 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4007740 con 0x7fb8ac06bb90 2024-04-24T22:39:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8a9600640 1 -- 172.21.15.47:0/3982329468 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a40078e0 con 0x7fb8ac06bb90 2024-04-24T22:39:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8a9600640 1 -- 172.21.15.47:0/3982329468 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4007cb0 con 0x7fb8ac06bb90 2024-04-24T22:39:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8abe00640 1 -- 172.21.15.47:0/3982329468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8ac06bb90 msgr2=0x7fb8ac15b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8abe00640 1 --2- 172.21.15.47:0/3982329468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8ac06bb90 0x7fb8ac15b860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac158b60 tx=0x7fb8a402a680 comp rx=0 tx=0).stop 2024-04-24T22:39:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8abe00640 1 -- 172.21.15.47:0/3982329468 shutdown_connections 2024-04-24T22:39:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8abe00640 1 --2- 172.21.15.47:0/3982329468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8ac06bb90 0x7fb8ac15b860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8abe00640 1 --2- 172.21.15.47:0/3982329468 >> v2:172.21.15.47:3300/0 conn(0x7fb8ac072c60 0x7fb8ac0c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8abe00640 1 -- 172.21.15.47:0/3982329468 >> 172.21.15.47:0/3982329468 conn(0x7fb8ac0cb900 msgr2=0x7fb8ac0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8abe00640 1 -- 172.21.15.47:0/3982329468 shutdown_connections 2024-04-24T22:39:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.956+0000 7fb8abe00640 1 -- 172.21.15.47:0/3982329468 wait complete. 2024-04-24T22:39:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.958+0000 7fb8abe00640 1 Processor -- start 2024-04-24T22:39:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.958+0000 7fb8abe00640 1 -- start start 2024-04-24T22:39:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.959+0000 7fb8abe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8ac06bb90 0x7fb8ac1e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.959+0000 7fb8abe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8ac072c60 0x7fb8ac13e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.959+0000 7fb8abe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8ac073040 0x7fb8ac1e2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.959+0000 7fb8abe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac0c32a0 con 0x7fb8ac06bb90 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.959+0000 7fb8abe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac158270 con 0x7fb8ac073040 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.959+0000 7fb8abe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb8ac159080 con 0x7fb8ac072c60 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.959+0000 7fb8aaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8ac06bb90 0x7fb8ac1e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.959+0000 7fb8aaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8ac06bb90 0x7fb8ac1e4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53586/0 (socket says 172.21.15.47:53586) 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.959+0000 7fb8aaa00640 1 -- 172.21.15.47:0/3720502496 learned_addr learned my addr 172.21.15.47:0/3720502496 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.960+0000 7fb8aa000640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8ac072c60 0x7fb8ac13e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.960+0000 7fb8ab400640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8ac073040 0x7fb8ac1e2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.960+0000 7fb8aaa00640 1 -- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8ac072c60 msgr2=0x7fb8ac13e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.960+0000 7fb8aaa00640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8ac072c60 0x7fb8ac13e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.960+0000 7fb8aaa00640 1 -- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8ac073040 msgr2=0x7fb8ac1e2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.960+0000 7fb8aaa00640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8ac073040 0x7fb8ac1e2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.960+0000 7fb8aaa00640 1 -- 172.21.15.47:0/3720502496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac1f2430 con 0x7fb8ac06bb90 2024-04-24T22:39:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.960+0000 7fb8aaa00640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8ac06bb90 0x7fb8ac1e4a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00e9b0 tx=0x7fb89c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.960+0000 7fb8a2a00640 1 -- 172.21.15.47:0/3720502496 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c017070 con 0x7fb8ac06bb90 2024-04-24T22:39:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.961+0000 7fb8a2a00640 1 -- 172.21.15.47:0/3720502496 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb89c00c9a0 con 0x7fb8ac06bb90 2024-04-24T22:39:41.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.961+0000 7fb8abe00640 1 -- 172.21.15.47:0/3720502496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ac0c81e0 con 0x7fb8ac06bb90 2024-04-24T22:39:41.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.961+0000 7fb8a2a00640 1 -- 172.21.15.47:0/3720502496 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00cd90 con 0x7fb8ac06bb90 2024-04-24T22:39:41.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.961+0000 7fb8abe00640 1 -- 172.21.15.47:0/3720502496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ac0c87a0 con 0x7fb8ac06bb90 2024-04-24T22:39:41.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.962+0000 7fb8abe00640 1 -- 172.21.15.47:0/3720502496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ac068d50 con 0x7fb8ac06bb90 2024-04-24T22:39:41.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.963+0000 7fb8a2a00640 1 -- 172.21.15.47:0/3720502496 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb89c01a070 con 0x7fb8ac06bb90 2024-04-24T22:39:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.964+0000 7fb8a2a00640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb88c06be30 0x7fb88c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.964+0000 7fb8a2a00640 1 -- 172.21.15.47:0/3720502496 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb89c08ae00 con 0x7fb8ac06bb90 2024-04-24T22:39:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.964+0000 7fb8aa000640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb88c06be30 0x7fb88c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.965+0000 7fb8aa000640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb88c06be30 0x7fb88c06e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0cb3e0 tx=0x7fb898007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:41.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:41.968+0000 7fb8a2a00640 1 -- 172.21.15.47:0/3720502496 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb89c059150 con 0x7fb8ac06bb90 2024-04-24T22:39:42.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.212+0000 7fb8abe00640 1 -- 172.21.15.47:0/3720502496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb8ac152940 con 0x7fb8ac06bb90 2024-04-24T22:39:42.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.724+0000 7fb8a2a00640 1 -- 172.21.15.47:0/3720502496 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7fb89c05d450 con 0x7fb8ac06bb90 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 -- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb88c06be30 msgr2=0x7fb88c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb88c06be30 0x7fb88c06e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0cb3e0 tx=0x7fb898007d50 comp rx=0 tx=0).stop 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 -- 172.21.15.47:0/3720502496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8ac06bb90 msgr2=0x7fb8ac1e4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8ac06bb90 0x7fb8ac1e4a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00e9b0 tx=0x7fb89c00ee70 comp rx=0 tx=0).stop 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 -- 172.21.15.47:0/3720502496 shutdown_connections 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb88c06be30 0x7fb88c06e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8ac073040 0x7fb8ac1e2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8ac072c60 0x7fb8ac13e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 --2- 172.21.15.47:0/3720502496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8ac06bb90 0x7fb8ac1e4a40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 -- 172.21.15.47:0/3720502496 >> 172.21.15.47:0/3720502496 conn(0x7fb8ac0cb900 msgr2=0x7fb8ac157630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:42.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.729+0000 7fb8abe00640 1 -- 172.21.15.47:0/3720502496 shutdown_connections 2024-04-24T22:39:42.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.730+0000 7fb8abe00640 1 -- 172.21.15.47:0/3720502496 wait complete. 2024-04-24T22:39:42.751 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T22:39:42.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:39:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.941+0000 7f6688200640 1 Processor -- start 2024-04-24T22:39:42.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.941+0000 7f6688200640 1 -- start start 2024-04-24T22:39:42.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.943+0000 7f6688200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6680158be0 0x7f66800c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.943+0000 7f6688200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6680158fc0 0x7f66800c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.943+0000 7f6688200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66801621d0 con 0x7f66800c1280 2024-04-24T22:39:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.943+0000 7f6688200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f66801624b0 con 0x7f6680158fc0 2024-04-24T22:39:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.943+0000 7f6688200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6680162790 con 0x7f6680158be0 2024-04-24T22:39:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.943+0000 7f6686200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f66800c1280 0x7f668015ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54418/0 (socket says 172.21.15.47:54418) 2024-04-24T22:39:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.943+0000 7f6684e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6680158fc0 0x7f66800c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.943+0000 7f6685800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6680158be0 0x7f66800c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:42.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.943+0000 7f6686200640 1 -- 172.21.15.47:0/1059029198 learned_addr learned my addr 172.21.15.47:0/1059029198 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.944+0000 7f6684e00640 1 -- 172.21.15.47:0/1059029198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6680158be0 msgr2=0x7f66800c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.944+0000 7f6684e00640 1 --2- 172.21.15.47:0/1059029198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6680158be0 0x7f66800c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.944+0000 7f6684e00640 1 -- 172.21.15.47:0/1059029198 >> v1:172.21.15.47:6789/0 conn(0x7f66800c1280 legacy=0x7f668015ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.944+0000 7f6684e00640 1 -- 172.21.15.47:0/1059029198 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6680162a70 con 0x7f6680158fc0 2024-04-24T22:39:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.945+0000 7f6684e00640 1 --2- 172.21.15.47:0/1059029198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6680158fc0 0x7f66800c8590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f66740097a0 tx=0x7f667402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5e6b25701f860a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.945+0000 7f667fe00640 1 -- 172.21.15.47:0/1059029198 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674037070 con 0x7f6680158fc0 2024-04-24T22:39:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.945+0000 7f667fe00640 1 -- 172.21.15.47:0/1059029198 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6674032440 con 0x7f6680158fc0 2024-04-24T22:39:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.945+0000 7f667fe00640 1 -- 172.21.15.47:0/1059029198 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674032810 con 0x7f6680158fc0 2024-04-24T22:39:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.946+0000 7f6688200640 1 -- 172.21.15.47:0/1059029198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6680158fc0 msgr2=0x7f66800c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.946+0000 7f6688200640 1 --2- 172.21.15.47:0/1059029198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6680158fc0 0x7f66800c8590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f66740097a0 tx=0x7f667402a210 comp rx=0 tx=0).stop 2024-04-24T22:39:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.946+0000 7f6688200640 1 -- 172.21.15.47:0/1059029198 shutdown_connections 2024-04-24T22:39:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.946+0000 7f6688200640 1 --2- 172.21.15.47:0/1059029198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6680158fc0 0x7f66800c8590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.946+0000 7f6688200640 1 --2- 172.21.15.47:0/1059029198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6680158be0 0x7f66800c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.946+0000 7f6688200640 1 -- 172.21.15.47:0/1059029198 >> 172.21.15.47:0/1059029198 conn(0x7f668006e580 msgr2=0x7f668006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.946+0000 7f6688200640 1 -- 172.21.15.47:0/1059029198 shutdown_connections 2024-04-24T22:39:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.946+0000 7f6688200640 1 -- 172.21.15.47:0/1059029198 wait complete. 2024-04-24T22:39:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.947+0000 7f6688200640 1 Processor -- start 2024-04-24T22:39:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.948+0000 7f6688200640 1 -- start start 2024-04-24T22:39:42.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.948+0000 7f6688200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800c1280 0x7f66800c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:42.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.948+0000 7f6688200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6680158be0 0x7f6680153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:42.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6688200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6680158fc0 0x7f66801f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6688200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f66800c1050 con 0x7f6680158fc0 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6688200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f66800c0ed0 con 0x7f6680158be0 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6688200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f668015f2e0 con 0x7f66800c1280 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6686200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6680158fc0 0x7f66801f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6685800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800c1280 0x7f66800c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6685800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800c1280 0x7f66800c2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:55816/0 (socket says 172.21.15.47:55816) 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6685800640 1 -- 172.21.15.47:0/2344516765 learned_addr learned my addr 172.21.15.47:0/2344516765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6684e00640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6680158be0 0x7f6680153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6685800640 1 -- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6680158be0 msgr2=0x7f6680153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6685800640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6680158be0 0x7f6680153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6685800640 1 -- 172.21.15.47:0/2344516765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6680158fc0 msgr2=0x7f66801f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6685800640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6680158fc0 0x7f66801f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:42.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6685800640 1 -- 172.21.15.47:0/2344516765 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6680160be0 con 0x7f66800c1280 2024-04-24T22:39:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.949+0000 7f6686200640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6680158fc0 0x7f66801f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.950+0000 7f6685800640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800c1280 0x7f66800c2f00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f666c00e9b0 tx=0x7f666c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.950+0000 7f667d600640 1 -- 172.21.15.47:0/2344516765 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c017070 con 0x7f66800c1280 2024-04-24T22:39:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.950+0000 7f667d600640 1 -- 172.21.15.47:0/2344516765 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666c00c9a0 con 0x7f66800c1280 2024-04-24T22:39:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.950+0000 7f667d600640 1 -- 172.21.15.47:0/2344516765 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c00cd90 con 0x7f66800c1280 2024-04-24T22:39:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.950+0000 7f6688200640 1 -- 172.21.15.47:0/2344516765 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6680160f40 con 0x7f66800c1280 2024-04-24T22:39:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.950+0000 7f6688200640 1 -- 172.21.15.47:0/2344516765 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f668014f9d0 con 0x7f66800c1280 2024-04-24T22:39:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.951+0000 7f6688200640 1 -- 172.21.15.47:0/2344516765 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66800b21b0 con 0x7f66800c1280 2024-04-24T22:39:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.952+0000 7f667d600640 1 -- 172.21.15.47:0/2344516765 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f666c01a070 con 0x7f66800c1280 2024-04-24T22:39:42.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.953+0000 7f667d600640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f666006be30 0x7f666006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:42.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.953+0000 7f667d600640 1 -- 172.21.15.47:0/2344516765 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f666c08af30 con 0x7f66800c1280 2024-04-24T22:39:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.953+0000 7f6684e00640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f666006be30 0x7f666006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.954+0000 7f6684e00640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f666006be30 0x7f666006e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6674034000 tx=0x7f667402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:42.958+0000 7f667d600640 1 -- 172.21.15.47:0/2344516765 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f66800b21b0 con 0x7f66800c1280 2024-04-24T22:39:43.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.201+0000 7f6688200640 1 -- 172.21.15.47:0/2344516765 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6680073b90 con 0x7f66800c1280 2024-04-24T22:39:43.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.202+0000 7f667d600640 1 -- 172.21.15.47:0/2344516765 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6828 (secure 0 0 0) 0x7f666c059200 con 0x7f66800c1280 2024-04-24T22:39:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:43.208 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1712577517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1712577517},{"type":"v1","addr":"172.21.15.113:6839","nonce":1712577517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2908847255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2908847255},{"type":"v1","addr":"172.21.15.47:6833","nonce":2908847255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2173854280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2173854280},{"type":"v1","addr":"172.21.15.47:6837","nonce":2173854280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3103107210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3103107210},{"type":"v1","addr":"172.21.15.113:6837","nonce":3103107210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6838/3621458364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3621458364},{"type":"v1","addr":"172.21.15.113:6838","nonce":3621458364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:39:37.178621+0000","modified":"2024-04-24T22:39:42.214184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"c","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.47:6835/1117134154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1117134154},{"type":"v1","addr":"172.21.15.47:6835","nonce":1117134154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 8 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.205+0000 7f6688200640 1 -- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f666006be30 msgr2=0x7f666006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.205+0000 7f6688200640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f666006be30 0x7f666006e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6674034000 tx=0x7f667402e000 comp rx=0 tx=0).stop 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.205+0000 7f6688200640 1 -- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800c1280 msgr2=0x7f66800c2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.205+0000 7f6688200640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800c1280 0x7f66800c2f00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f666c00e9b0 tx=0x7f666c00ee70 comp rx=0 tx=0).stop 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.205+0000 7f6688200640 1 -- 172.21.15.47:0/2344516765 shutdown_connections 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.205+0000 7f6688200640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f666006be30 0x7f666006e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.205+0000 7f6688200640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6680158fc0 0x7f66801f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.205+0000 7f6688200640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6680158be0 0x7f6680153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.205+0000 7f6688200640 1 --2- 172.21.15.47:0/2344516765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800c1280 0x7f66800c2f00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.205+0000 7f6688200640 1 -- 172.21.15.47:0/2344516765 >> 172.21.15.47:0/2344516765 conn(0x7f668006e580 msgr2=0x7f668015ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:43.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.206+0000 7f6688200640 1 -- 172.21.15.47:0/2344516765 shutdown_connections 2024-04-24T22:39:43.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.206+0000 7f6688200640 1 -- 172.21.15.47:0/2344516765 wait complete. 2024-04-24T22:39:43.226 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:43.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.440+0000 7f2847200640 1 Processor -- start 2024-04-24T22:39:43.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.440+0000 7f2847200640 1 -- start start 2024-04-24T22:39:43.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.442+0000 7f2847200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2840152960 0x7f28400c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.442+0000 7f2847200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28400c8fa0 0x7f284015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.442+0000 7f2847200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28400c9400 con 0x7f284015bf60 2024-04-24T22:39:43.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.442+0000 7f2847200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2840158f20 con 0x7f2840152960 2024-04-24T22:39:43.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.442+0000 7f2847200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f28401593b0 con 0x7f28400c8fa0 2024-04-24T22:39:43.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.443+0000 7f2845200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f284015bf60 0x7f2840072b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54432/0 (socket says 172.21.15.47:54432) 2024-04-24T22:39:43.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.443+0000 7f2845200640 1 -- 172.21.15.47:0/3621916660 learned_addr learned my addr 172.21.15.47:0/3621916660 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:43.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.443+0000 7f283fe00640 1 --2- 172.21.15.47:0/3621916660 >> v2:172.21.15.47:3300/0 conn(0x7f2840152960 0x7f28400c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.443+0000 7f283f400640 1 --2- 172.21.15.47:0/3621916660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28400c8fa0 0x7f284015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.443+0000 7f283ea00640 1 -- 172.21.15.47:0/3621916660 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2653289515 0 0) 0x7f28400c9400 con 0x7f284015bf60 2024-04-24T22:39:43.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.443+0000 7f283ea00640 1 -- 172.21.15.47:0/3621916660 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2824003520 con 0x7f284015bf60 2024-04-24T22:39:43.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.443+0000 7f283f400640 1 -- 172.21.15.47:0/3621916660 >> v2:172.21.15.47:3300/0 conn(0x7f2840152960 msgr2=0x7f28400c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:43.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.443+0000 7f283f400640 1 --2- 172.21.15.47:0/3621916660 >> v2:172.21.15.47:3300/0 conn(0x7f2840152960 0x7f28400c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.443+0000 7f283f400640 1 -- 172.21.15.47:0/3621916660 >> v1:172.21.15.47:6789/0 conn(0x7f284015bf60 legacy=0x7f2840072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.444+0000 7f283f400640 1 -- 172.21.15.47:0/3621916660 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28400c7e50 con 0x7f28400c8fa0 2024-04-24T22:39:43.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.445+0000 7f283f400640 1 --2- 172.21.15.47:0/3621916660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28400c8fa0 0x7f284015ba20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f282c0097a0 tx=0x7f282c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f105ef0c350e06d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:43.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.445+0000 7f283ea00640 1 -- 172.21.15.47:0/3621916660 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c037070 con 0x7f28400c8fa0 2024-04-24T22:39:43.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.445+0000 7f283ea00640 1 -- 172.21.15.47:0/3621916660 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c032440 con 0x7f28400c8fa0 2024-04-24T22:39:43.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.445+0000 7f283ea00640 1 -- 172.21.15.47:0/3621916660 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c032810 con 0x7f28400c8fa0 2024-04-24T22:39:43.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.446+0000 7f2847200640 1 -- 172.21.15.47:0/3621916660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28400c8fa0 msgr2=0x7f284015ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.446+0000 7f2847200640 1 --2- 172.21.15.47:0/3621916660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28400c8fa0 0x7f284015ba20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f282c0097a0 tx=0x7f282c02a210 comp rx=0 tx=0).stop 2024-04-24T22:39:43.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.446+0000 7f2847200640 1 -- 172.21.15.47:0/3621916660 shutdown_connections 2024-04-24T22:39:43.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.446+0000 7f2847200640 1 --2- 172.21.15.47:0/3621916660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28400c8fa0 0x7f284015ba20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.446+0000 7f2847200640 1 --2- 172.21.15.47:0/3621916660 >> v2:172.21.15.47:3300/0 conn(0x7f2840152960 0x7f28400c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.446+0000 7f2847200640 1 -- 172.21.15.47:0/3621916660 >> 172.21.15.47:0/3621916660 conn(0x7f284006e580 msgr2=0x7f284006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:43.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.446+0000 7f2847200640 1 -- 172.21.15.47:0/3621916660 shutdown_connections 2024-04-24T22:39:43.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.446+0000 7f2847200640 1 -- 172.21.15.47:0/3621916660 wait complete. 2024-04-24T22:39:43.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.448+0000 7f2847200640 1 Processor -- start 2024-04-24T22:39:43.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.448+0000 7f2847200640 1 -- start start 2024-04-24T22:39:43.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f2847200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28400c8fa0 0x7f28401e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f2847200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2840152960 0x7f28401e2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f2847200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f284015bf60 0x7f28401f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f2847200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f28401588d0 con 0x7f2840152960 2024-04-24T22:39:43.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f2847200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f28400c8130 con 0x7f28400c8fa0 2024-04-24T22:39:43.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f2847200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2840158bb0 con 0x7f284015bf60 2024-04-24T22:39:43.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f283f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2840152960 0x7f28401e2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f283f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2840152960 0x7f28401e2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53626/0 (socket says 172.21.15.47:53626) 2024-04-24T22:39:43.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f283fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28400c8fa0 0x7f28401e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f283f400640 1 -- 172.21.15.47:0/192027104 learned_addr learned my addr 172.21.15.47:0/192027104 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:43.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f2845200640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f284015bf60 0x7f28401f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f283f400640 1 -- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f284015bf60 msgr2=0x7f28401f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f283f400640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f284015bf60 0x7f28401f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f283f400640 1 -- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28400c8fa0 msgr2=0x7f28401e50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f283f400640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28400c8fa0 0x7f28401e50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f283f400640 1 -- 172.21.15.47:0/192027104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28401e36b0 con 0x7f2840152960 2024-04-24T22:39:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.450+0000 7f2845200640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f284015bf60 0x7f28401f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.451+0000 7f283f400640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2840152960 0x7f28401e2c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f282c035040 tx=0x7f282c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.451+0000 7f2837e00640 1 -- 172.21.15.47:0/192027104 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c041030 con 0x7f2840152960 2024-04-24T22:39:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.451+0000 7f2837e00640 1 -- 172.21.15.47:0/192027104 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c03d4e0 con 0x7f2840152960 2024-04-24T22:39:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.451+0000 7f2837e00640 1 -- 172.21.15.47:0/192027104 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c037040 con 0x7f2840152960 2024-04-24T22:39:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.451+0000 7f2847200640 1 -- 172.21.15.47:0/192027104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284015bc00 con 0x7f2840152960 2024-04-24T22:39:43.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.451+0000 7f2847200640 1 -- 172.21.15.47:0/192027104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28401e3c00 con 0x7f2840152960 2024-04-24T22:39:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.453+0000 7f2847200640 1 -- 172.21.15.47:0/192027104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2840073b90 con 0x7f2840152960 2024-04-24T22:39:43.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.454+0000 7f2837e00640 1 -- 172.21.15.47:0/192027104 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f282c044070 con 0x7f2840152960 2024-04-24T22:39:43.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.454+0000 7f2837e00640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f282006be30 0x7f282006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.455+0000 7f2837e00640 1 -- 172.21.15.47:0/192027104 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f282c0ab4a0 con 0x7f2840152960 2024-04-24T22:39:43.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.455+0000 7f283fe00640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f282006be30 0x7f282006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.456+0000 7f283fe00640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f282006be30 0x7f282006e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2830009b40 tx=0x7f2830009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:43.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.459+0000 7f2837e00640 1 -- 172.21.15.47:0/192027104 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f282c079e60 con 0x7f2840152960 2024-04-24T22:39:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.661+0000 7f2847200640 1 -- 172.21.15.47:0/192027104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f284006c200 con 0x7f2840152960 2024-04-24T22:39:43.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.663+0000 7f2837e00640 1 -- 172.21.15.47:0/192027104 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16593 (secure 0 0 0) 0x7f282c079760 con 0x7f2840152960 2024-04-24T22:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:43.665 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:38.692613+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 -- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f282006be30 msgr2=0x7f282006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f282006be30 0x7f282006e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2830009b40 tx=0x7f2830009190 comp rx=0 tx=0).stop 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 -- 172.21.15.47:0/192027104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2840152960 msgr2=0x7f28401e2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2840152960 0x7f28401e2c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f282c035040 tx=0x7f282c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 -- 172.21.15.47:0/192027104 shutdown_connections 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f282006be30 0x7f282006e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f284015bf60 0x7f28401f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2840152960 0x7f28401e2c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 --2- 172.21.15.47:0/192027104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28400c8fa0 0x7f28401e50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 -- 172.21.15.47:0/192027104 >> 172.21.15.47:0/192027104 conn(0x7f284006e580 msgr2=0x7f28400c71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 -- 172.21.15.47:0/192027104 shutdown_connections 2024-04-24T22:39:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.666+0000 7f2847200640 1 -- 172.21.15.47:0/192027104 wait complete. 2024-04-24T22:39:43.687 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T22:39:43.687 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:39:43.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.900+0000 7f2ccd000640 1 Processor -- start 2024-04-24T22:39:43.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.900+0000 7f2ccd000640 1 -- start start 2024-04-24T22:39:43.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.902+0000 7f2ccd000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2cc8072c60 0x7f2cc80c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.902+0000 7f2ccd000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc806bb90 0x7f2cc815b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.902+0000 7f2ccd000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cc815c3d0 con 0x7f2cc8073040 2024-04-24T22:39:43.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.902+0000 7f2ccd000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc8158bf0 con 0x7f2cc8072c60 2024-04-24T22:39:43.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.902+0000 7f2ccd000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc815c6b0 con 0x7f2cc806bb90 2024-04-24T22:39:43.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.902+0000 7f2cc7400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2cc8072c60 0x7f2cc80c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.902+0000 7f2cc7400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2cc8072c60 0x7f2cc80c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53634/0 (socket says 172.21.15.47:53634) 2024-04-24T22:39:43.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.902+0000 7f2cc7400640 1 -- 172.21.15.47:0/1588761689 learned_addr learned my addr 172.21.15.47:0/1588761689 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:43.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.902+0000 7f2cc7e00640 1 --2- 172.21.15.47:0/1588761689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc806bb90 0x7f2cc815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.903+0000 7f2cc7400640 1 -- 172.21.15.47:0/1588761689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc806bb90 msgr2=0x7f2cc815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:43.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.903+0000 7f2cc7400640 1 --2- 172.21.15.47:0/1588761689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc806bb90 0x7f2cc815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.903+0000 7f2cc7400640 1 -- 172.21.15.47:0/1588761689 >> v1:172.21.15.113:6789/0 conn(0x7f2cc8073040 legacy=0x7f2cc81411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.903+0000 7f2cc7400640 1 -- 172.21.15.47:0/1588761689 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc81585a0 con 0x7f2cc8072c60 2024-04-24T22:39:43.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.903+0000 7f2cc7e00640 1 --2- 172.21.15.47:0/1588761689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc806bb90 0x7f2cc815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:43.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.904+0000 7f2cc7400640 1 --2- 172.21.15.47:0/1588761689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc8072c60 0x7f2cc80c2d60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009870 tx=0x7f2cb802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7638f1464fee9364 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:43.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.904+0000 7f2cc6000640 1 -- 172.21.15.47:0/1588761689 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8037070 con 0x7f2cc8072c60 2024-04-24T22:39:43.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.904+0000 7f2cc6000640 1 -- 172.21.15.47:0/1588761689 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb8032440 con 0x7f2cc8072c60 2024-04-24T22:39:43.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.904+0000 7f2cc6000640 1 -- 172.21.15.47:0/1588761689 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8032810 con 0x7f2cc8072c60 2024-04-24T22:39:43.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.905+0000 7f2ccd000640 1 -- 172.21.15.47:0/1588761689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc8072c60 msgr2=0x7f2cc80c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.905+0000 7f2ccd000640 1 --2- 172.21.15.47:0/1588761689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc8072c60 0x7f2cc80c2d60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009870 tx=0x7f2cb802a250 comp rx=0 tx=0).stop 2024-04-24T22:39:43.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.905+0000 7f2ccd000640 1 -- 172.21.15.47:0/1588761689 shutdown_connections 2024-04-24T22:39:43.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.905+0000 7f2ccd000640 1 --2- 172.21.15.47:0/1588761689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc806bb90 0x7f2cc815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.905+0000 7f2ccd000640 1 --2- 172.21.15.47:0/1588761689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc8072c60 0x7f2cc80c2d60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.905+0000 7f2ccd000640 1 -- 172.21.15.47:0/1588761689 >> 172.21.15.47:0/1588761689 conn(0x7f2cc80cb900 msgr2=0x7f2cc80cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:43.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.905+0000 7f2ccd000640 1 -- 172.21.15.47:0/1588761689 shutdown_connections 2024-04-24T22:39:43.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.905+0000 7f2ccd000640 1 -- 172.21.15.47:0/1588761689 wait complete. 2024-04-24T22:39:43.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.907+0000 7f2ccd000640 1 Processor -- start 2024-04-24T22:39:43.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.907+0000 7f2ccd000640 1 -- start start 2024-04-24T22:39:43.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2ccd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc806bb90 0x7f2cc80c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2ccd000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc8072c60 0x7f2cc80c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2ccd000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc8073040 0x7f2cc80c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2ccd000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc80c32a0 con 0x7f2cc806bb90 2024-04-24T22:39:43.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2ccd000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8158270 con 0x7f2cc8073040 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2ccd000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc8159080 con 0x7f2cc8072c60 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2cc7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc806bb90 0x7f2cc80c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2cc7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc806bb90 0x7f2cc80c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53650/0 (socket says 172.21.15.47:53650) 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2cc7400640 1 -- 172.21.15.47:0/2068699778 learned_addr learned my addr 172.21.15.47:0/2068699778 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2cc6a00640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc8072c60 0x7f2cc80c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2cc7e00640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc8073040 0x7f2cc80c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2cc7400640 1 -- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc8072c60 msgr2=0x7f2cc80c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.908+0000 7f2cc7400640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc8072c60 0x7f2cc80c18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.909+0000 7f2cc7400640 1 -- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc8073040 msgr2=0x7f2cc80c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.909+0000 7f2cc7400640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc8073040 0x7f2cc80c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:43.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.909+0000 7f2cc7400640 1 -- 172.21.15.47:0/2068699778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc80c0b80 con 0x7f2cc806bb90 2024-04-24T22:39:43.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.909+0000 7f2cc7400640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc806bb90 0x7f2cc80c8680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2cb802a290 tx=0x7f2cb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:43.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.909+0000 7f2cbf400640 1 -- 172.21.15.47:0/2068699778 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8041020 con 0x7f2cc806bb90 2024-04-24T22:39:43.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.909+0000 7f2cbf400640 1 -- 172.21.15.47:0/2068699778 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb80306c0 con 0x7f2cc806bb90 2024-04-24T22:39:43.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.909+0000 7f2cbf400640 1 -- 172.21.15.47:0/2068699778 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8037040 con 0x7f2cc806bb90 2024-04-24T22:39:43.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.909+0000 7f2ccd000640 1 -- 172.21.15.47:0/2068699778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc81f0ef0 con 0x7f2cc806bb90 2024-04-24T22:39:43.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.909+0000 7f2ccd000640 1 -- 172.21.15.47:0/2068699778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc81f11d0 con 0x7f2cc806bb90 2024-04-24T22:39:43.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.911+0000 7f2ccd000640 1 -- 172.21.15.47:0/2068699778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc8068d50 con 0x7f2cc806bb90 2024-04-24T22:39:43.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.911+0000 7f2cbf400640 1 -- 172.21.15.47:0/2068699778 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2cb8044020 con 0x7f2cc806bb90 2024-04-24T22:39:43.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.912+0000 7f2cbf400640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ca806be60 0x7f2ca806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:43.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.913+0000 7f2cbf400640 1 -- 172.21.15.47:0/2068699778 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2cb80ab230 con 0x7f2cc806bb90 2024-04-24T22:39:43.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.913+0000 7f2cc6a00640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ca806be60 0x7f2ca806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:43.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.913+0000 7f2cc6a00640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ca806be60 0x7f2ca806e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80c93c0 tx=0x7f2cb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:43.918+0000 7f2cbf400640 1 -- 172.21.15.47:0/2068699778 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2cb804b070 con 0x7f2cc806bb90 2024-04-24T22:39:44.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.116+0000 7f2ccd000640 1 -- 172.21.15.47:0/2068699778 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cc804f5e0 con 0x7f2ca806be60 2024-04-24T22:39:44.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.125+0000 7f2cbf400640 1 -- 172.21.15.47:0/2068699778 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251845 (secure 0 0 0) 0x7f2cc804f5e0 con 0x7f2ca806be60 2024-04-24T22:39:44.131 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:44.133 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:39:44.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 -- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ca806be60 msgr2=0x7f2ca806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ca806be60 0x7f2ca806e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80c93c0 tx=0x7f2cb4009040 comp rx=0 tx=0).stop 2024-04-24T22:39:44.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 -- 172.21.15.47:0/2068699778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc806bb90 msgr2=0x7f2cc80c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc806bb90 0x7f2cc80c8680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2cb802a290 tx=0x7f2cb80046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 -- 172.21.15.47:0/2068699778 shutdown_connections 2024-04-24T22:39:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ca806be60 0x7f2ca806e320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cc8073040 0x7f2cc80c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cc8072c60 0x7f2cc80c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 --2- 172.21.15.47:0/2068699778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cc806bb90 0x7f2cc80c8680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 -- 172.21.15.47:0/2068699778 >> 172.21.15.47:0/2068699778 conn(0x7f2cc80cb900 msgr2=0x7f2cc80cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 -- 172.21.15.47:0/2068699778 shutdown_connections 2024-04-24T22:39:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.130+0000 7f2ccd000640 1 -- 172.21.15.47:0/2068699778 wait complete. 2024-04-24T22:39:44.153 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-24T22:39:42.642959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235708,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739012,"statfs":{"total":773094113280,"available":772852748288,"internally_reserved":0,"allocated":2924544,"data_stored":1785576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001379"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706276+0000","last_change":"2024-04-24T22:39:37.699192+0000","last_active":"2024-04-24T22:39:38.706276+0000","last_peered":"2024-04-24T22:39:38.706276+0000","last_clean":"2024-04-24T22:39:38.706276+0000","last_became_active":"2024-04-24T22:39:37.698920+0000","last_became_peered":"2024-04-24T22:39:37.698920+0000","last_unstale":"2024-04-24T22:39:38.706276+0000","last_undegraded":"2024-04-24T22:39:38.706276+0000","last_fullsized":"2024-04-24T22:39:38.706276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.821236+0000","last_change":"2024-04-24T22:39:37.697245+0000","last_active":"2024-04-24T22:39:38.821236+0000","last_peered":"2024-04-24T22:39:38.821236+0000","last_clean":"2024-04-24T22:39:38.821236+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:39:38.821236+0000","last_undegraded":"2024-04-24T22:39:38.821236+0000","last_fullsized":"2024-04-24T22:39:38.821236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705212+0000","last_change":"2024-04-24T22:39:37.709539+0000","last_active":"2024-04-24T22:39:38.705212+0000","last_peered":"2024-04-24T22:39:38.705212+0000","last_clean":"2024-04-24T22:39:38.705212+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:39:38.705212+0000","last_undegraded":"2024-04-24T22:39:38.705212+0000","last_fullsized":"2024-04-24T22:39:38.705212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820523+0000","last_change":"2024-04-24T22:39:37.719136+0000","last_active":"2024-04-24T22:39:38.820523+0000","last_peered":"2024-04-24T22:39:38.820523+0000","last_clean":"2024-04-24T22:39:38.820523+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:39:38.820523+0000","last_undegraded":"2024-04-24T22:39:38.820523+0000","last_fullsized":"2024-04-24T22:39:38.820523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705940+0000","last_change":"2024-04-24T22:39:37.704671+0000","last_active":"2024-04-24T22:39:38.705940+0000","last_peered":"2024-04-24T22:39:38.705940+0000","last_clean":"2024-04-24T22:39:38.705940+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:39:38.705940+0000","last_undegraded":"2024-04-24T22:39:38.705940+0000","last_fullsized":"2024-04-24T22:39:38.705940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820279+0000","last_change":"2024-04-24T22:39:37.703880+0000","last_active":"2024-04-24T22:39:38.820279+0000","last_peered":"2024-04-24T22:39:38.820279+0000","last_clean":"2024-04-24T22:39:38.820279+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:39:38.820279+0000","last_undegraded":"2024-04-24T22:39:38.820279+0000","last_fullsized":"2024-04-24T22:39:38.820279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705748+0000","last_change":"2024-04-24T22:39:37.703539+0000","last_active":"2024-04-24T22:39:38.705748+0000","last_peered":"2024-04-24T22:39:38.705748+0000","last_clean":"2024-04-24T22:39:38.705748+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:39:38.705748+0000","last_undegraded":"2024-04-24T22:39:38.705748+0000","last_fullsized":"2024-04-24T22:39:38.705748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707950+0000","last_change":"2024-04-24T22:39:37.713543+0000","last_active":"2024-04-24T22:39:38.707950+0000","last_peered":"2024-04-24T22:39:38.707950+0000","last_clean":"2024-04-24T22:39:38.707950+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:39:38.707950+0000","last_undegraded":"2024-04-24T22:39:38.707950+0000","last_fullsized":"2024-04-24T22:39:38.707950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706353+0000","last_change":"2024-04-24T22:39:37.706715+0000","last_active":"2024-04-24T22:39:38.706353+0000","last_peered":"2024-04-24T22:39:38.706353+0000","last_clean":"2024-04-24T22:39:38.706353+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:39:38.706353+0000","last_undegraded":"2024-04-24T22:39:38.706353+0000","last_fullsized":"2024-04-24T22:39:38.706353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819558+0000","last_change":"2024-04-24T22:39:37.701250+0000","last_active":"2024-04-24T22:39:38.819558+0000","last_peered":"2024-04-24T22:39:38.819558+0000","last_clean":"2024-04-24T22:39:38.819558+0000","last_became_active":"2024-04-24T22:39:37.700959+0000","last_became_peered":"2024-04-24T22:39:37.700959+0000","last_unstale":"2024-04-24T22:39:38.819558+0000","last_undegraded":"2024-04-24T22:39:38.819558+0000","last_fullsized":"2024-04-24T22:39:38.819558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819186+0000","last_change":"2024-04-24T22:39:37.701069+0000","last_active":"2024-04-24T22:39:38.819186+0000","last_peered":"2024-04-24T22:39:38.819186+0000","last_clean":"2024-04-24T22:39:38.819186+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:39:38.819186+0000","last_undegraded":"2024-04-24T22:39:38.819186+0000","last_fullsized":"2024-04-24T22:39:38.819186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819923+0000","last_change":"2024-04-24T22:39:37.693508+0000","last_active":"2024-04-24T22:39:38.819923+0000","last_peered":"2024-04-24T22:39:38.819923+0000","last_clean":"2024-04-24T22:39:38.819923+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:39:38.819923+0000","last_undegraded":"2024-04-24T22:39:38.819923+0000","last_fullsized":"2024-04-24T22:39:38.819923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708270+0000","last_change":"2024-04-24T22:39:37.705157+0000","last_active":"2024-04-24T22:39:38.708270+0000","last_peered":"2024-04-24T22:39:38.708270+0000","last_clean":"2024-04-24T22:39:38.708270+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:39:38.708270+0000","last_undegraded":"2024-04-24T22:39:38.708270+0000","last_fullsized":"2024-04-24T22:39:38.708270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704755+0000","last_change":"2024-04-24T22:39:37.698333+0000","last_active":"2024-04-24T22:39:38.704755+0000","last_peered":"2024-04-24T22:39:38.704755+0000","last_clean":"2024-04-24T22:39:38.704755+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:39:38.704755+0000","last_undegraded":"2024-04-24T22:39:38.704755+0000","last_fullsized":"2024-04-24T22:39:38.704755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:57.510197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820219+0000","last_change":"2024-04-24T22:39:37.696874+0000","last_active":"2024-04-24T22:39:38.820219+0000","last_peered":"2024-04-24T22:39:38.820219+0000","last_clean":"2024-04-24T22:39:38.820219+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:39:38.820219+0000","last_undegraded":"2024-04-24T22:39:38.820219+0000","last_fullsized":"2024-04-24T22:39:38.820219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820021+0000","last_change":"2024-04-24T22:39:37.703446+0000","last_active":"2024-04-24T22:39:38.820021+0000","last_peered":"2024-04-24T22:39:38.820021+0000","last_clean":"2024-04-24T22:39:38.820021+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:39:38.820021+0000","last_undegraded":"2024-04-24T22:39:38.820021+0000","last_fullsized":"2024-04-24T22:39:38.820021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820971+0000","last_change":"2024-04-24T22:39:37.701515+0000","last_active":"2024-04-24T22:39:38.820971+0000","last_peered":"2024-04-24T22:39:38.820971+0000","last_clean":"2024-04-24T22:39:38.820971+0000","last_became_active":"2024-04-24T22:39:37.700439+0000","last_became_peered":"2024-04-24T22:39:37.700439+0000","last_unstale":"2024-04-24T22:39:38.820971+0000","last_undegraded":"2024-04-24T22:39:38.820971+0000","last_fullsized":"2024-04-24T22:39:38.820971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819628+0000","last_change":"2024-04-24T22:39:37.703899+0000","last_active":"2024-04-24T22:39:38.819628+0000","last_peered":"2024-04-24T22:39:38.819628+0000","last_clean":"2024-04-24T22:39:38.819628+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:39:38.819628+0000","last_undegraded":"2024-04-24T22:39:38.819628+0000","last_fullsized":"2024-04-24T22:39:38.819628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706244+0000","last_change":"2024-04-24T22:39:37.711019+0000","last_active":"2024-04-24T22:39:38.706244+0000","last_peered":"2024-04-24T22:39:38.706244+0000","last_clean":"2024-04-24T22:39:38.706244+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:39:38.706244+0000","last_undegraded":"2024-04-24T22:39:38.706244+0000","last_fullsized":"2024-04-24T22:39:38.706244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819357+0000","last_change":"2024-04-24T22:39:37.696214+0000","last_active":"2024-04-24T22:39:38.819357+0000","last_peered":"2024-04-24T22:39:38.819357+0000","last_clean":"2024-04-24T22:39:38.819357+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:39:38.819357+0000","last_undegraded":"2024-04-24T22:39:38.819357+0000","last_fullsized":"2024-04-24T22:39:38.819357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705029+0000","last_change":"2024-04-24T22:39:37.700151+0000","last_active":"2024-04-24T22:39:38.705029+0000","last_peered":"2024-04-24T22:39:38.705029+0000","last_clean":"2024-04-24T22:39:38.705029+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:39:38.705029+0000","last_undegraded":"2024-04-24T22:39:38.705029+0000","last_fullsized":"2024-04-24T22:39:38.705029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820267+0000","last_change":"2024-04-24T22:39:37.705083+0000","last_active":"2024-04-24T22:39:38.820267+0000","last_peered":"2024-04-24T22:39:38.820267+0000","last_clean":"2024-04-24T22:39:38.820267+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:39:38.820267+0000","last_undegraded":"2024-04-24T22:39:38.820267+0000","last_fullsized":"2024-04-24T22:39:38.820267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708194+0000","last_change":"2024-04-24T22:39:37.711193+0000","last_active":"2024-04-24T22:39:38.708194+0000","last_peered":"2024-04-24T22:39:38.708194+0000","last_clean":"2024-04-24T22:39:38.708194+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:39:38.708194+0000","last_undegraded":"2024-04-24T22:39:38.708194+0000","last_fullsized":"2024-04-24T22:39:38.708194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820128+0000","last_change":"2024-04-24T22:39:37.713910+0000","last_active":"2024-04-24T22:39:38.820128+0000","last_peered":"2024-04-24T22:39:38.820128+0000","last_clean":"2024-04-24T22:39:38.820128+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:39:38.820128+0000","last_undegraded":"2024-04-24T22:39:38.820128+0000","last_fullsized":"2024-04-24T22:39:38.820128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708266+0000","last_change":"2024-04-24T22:39:37.717191+0000","last_active":"2024-04-24T22:39:38.708266+0000","last_peered":"2024-04-24T22:39:38.708266+0000","last_clean":"2024-04-24T22:39:38.708266+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:39:38.708266+0000","last_undegraded":"2024-04-24T22:39:38.708266+0000","last_fullsized":"2024-04-24T22:39:38.708266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819555+0000","last_change":"2024-04-24T22:39:37.702983+0000","last_active":"2024-04-24T22:39:38.819555+0000","last_peered":"2024-04-24T22:39:38.819555+0000","last_clean":"2024-04-24T22:39:38.819555+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:39:38.819555+0000","last_undegraded":"2024-04-24T22:39:38.819555+0000","last_fullsized":"2024-04-24T22:39:38.819555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820583+0000","last_change":"2024-04-24T22:39:37.714111+0000","last_active":"2024-04-24T22:39:38.820583+0000","last_peered":"2024-04-24T22:39:38.820583+0000","last_clean":"2024-04-24T22:39:38.820583+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:39:38.820583+0000","last_undegraded":"2024-04-24T22:39:38.820583+0000","last_fullsized":"2024-04-24T22:39:38.820583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708923+0000","last_change":"2024-04-24T22:39:37.714877+0000","last_active":"2024-04-24T22:39:38.708923+0000","last_peered":"2024-04-24T22:39:38.708923+0000","last_clean":"2024-04-24T22:39:38.708923+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:39:38.708923+0000","last_undegraded":"2024-04-24T22:39:38.708923+0000","last_fullsized":"2024-04-24T22:39:38.708923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705236+0000","last_change":"2024-04-24T22:39:37.714695+0000","last_active":"2024-04-24T22:39:38.705236+0000","last_peered":"2024-04-24T22:39:38.705236+0000","last_clean":"2024-04-24T22:39:38.705236+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:39:38.705236+0000","last_undegraded":"2024-04-24T22:39:38.705236+0000","last_fullsized":"2024-04-24T22:39:38.705236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820461+0000","last_change":"2024-04-24T22:39:37.719187+0000","last_active":"2024-04-24T22:39:38.820461+0000","last_peered":"2024-04-24T22:39:38.820461+0000","last_clean":"2024-04-24T22:39:38.820461+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:39:38.820461+0000","last_undegraded":"2024-04-24T22:39:38.820461+0000","last_fullsized":"2024-04-24T22:39:38.820461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819723+0000","last_change":"2024-04-24T22:39:37.707283+0000","last_active":"2024-04-24T22:39:38.819723+0000","last_peered":"2024-04-24T22:39:38.819723+0000","last_clean":"2024-04-24T22:39:38.819723+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:39:38.819723+0000","last_undegraded":"2024-04-24T22:39:38.819723+0000","last_fullsized":"2024-04-24T22:39:38.819723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818968+0000","last_change":"2024-04-24T22:39:37.697902+0000","last_active":"2024-04-24T22:39:38.818968+0000","last_peered":"2024-04-24T22:39:38.818968+0000","last_clean":"2024-04-24T22:39:38.818968+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:39:38.818968+0000","last_undegraded":"2024-04-24T22:39:38.818968+0000","last_fullsized":"2024-04-24T22:39:38.818968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819142+0000","last_change":"2024-04-24T22:39:37.705667+0000","last_active":"2024-04-24T22:39:38.819142+0000","last_peered":"2024-04-24T22:39:38.819142+0000","last_clean":"2024-04-24T22:39:38.819142+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:39:38.819142+0000","last_undegraded":"2024-04-24T22:39:38.819142+0000","last_fullsized":"2024-04-24T22:39:38.819142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:50.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708783+0000","last_change":"2024-04-24T22:39:36.707990+0000","last_active":"2024-04-24T22:39:38.708783+0000","last_peered":"2024-04-24T22:39:38.708783+0000","last_clean":"2024-04-24T22:39:38.708783+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:39:38.708783+0000","last_undegraded":"2024-04-24T22:39:38.708783+0000","last_fullsized":"2024-04-24T22:39:38.708783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706420+0000","last_change":"2024-04-24T22:39:37.700406+0000","last_active":"2024-04-24T22:39:38.706420+0000","last_peered":"2024-04-24T22:39:38.706420+0000","last_clean":"2024-04-24T22:39:38.706420+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:39:38.706420+0000","last_undegraded":"2024-04-24T22:39:38.706420+0000","last_fullsized":"2024-04-24T22:39:38.706420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819221+0000","last_change":"2024-04-24T22:39:36.701398+0000","last_active":"2024-04-24T22:39:38.819221+0000","last_peered":"2024-04-24T22:39:38.819221+0000","last_clean":"2024-04-24T22:39:38.819221+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:39:38.819221+0000","last_undegraded":"2024-04-24T22:39:38.819221+0000","last_fullsized":"2024-04-24T22:39:38.819221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708466+0000","last_change":"2024-04-24T22:39:36.712294+0000","last_active":"2024-04-24T22:39:38.708466+0000","last_peered":"2024-04-24T22:39:38.708466+0000","last_clean":"2024-04-24T22:39:38.708466+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:39:38.708466+0000","last_undegraded":"2024-04-24T22:39:38.708466+0000","last_fullsized":"2024-04-24T22:39:38.708466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707809+0000","last_change":"2024-04-24T22:39:37.716920+0000","last_active":"2024-04-24T22:39:38.707809+0000","last_peered":"2024-04-24T22:39:38.707809+0000","last_clean":"2024-04-24T22:39:38.707809+0000","last_became_active":"2024-04-24T22:39:37.716641+0000","last_became_peered":"2024-04-24T22:39:37.716641+0000","last_unstale":"2024-04-24T22:39:38.707809+0000","last_undegraded":"2024-04-24T22:39:38.707809+0000","last_fullsized":"2024-04-24T22:39:38.707809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820143+0000","last_change":"2024-04-24T22:39:36.698983+0000","last_active":"2024-04-24T22:39:38.820143+0000","last_peered":"2024-04-24T22:39:38.820143+0000","last_clean":"2024-04-24T22:39:38.820143+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:39:38.820143+0000","last_undegraded":"2024-04-24T22:39:38.820143+0000","last_fullsized":"2024-04-24T22:39:38.820143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820135+0000","last_change":"2024-04-24T22:39:37.714229+0000","last_active":"2024-04-24T22:39:38.820135+0000","last_peered":"2024-04-24T22:39:38.820135+0000","last_clean":"2024-04-24T22:39:38.820135+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:39:38.820135+0000","last_undegraded":"2024-04-24T22:39:38.820135+0000","last_fullsized":"2024-04-24T22:39:38.820135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819388+0000","last_change":"2024-04-24T22:39:36.705776+0000","last_active":"2024-04-24T22:39:38.819388+0000","last_peered":"2024-04-24T22:39:38.819388+0000","last_clean":"2024-04-24T22:39:38.819388+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:39:38.819388+0000","last_undegraded":"2024-04-24T22:39:38.819388+0000","last_fullsized":"2024-04-24T22:39:38.819388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820936+0000","last_change":"2024-04-24T22:39:37.701436+0000","last_active":"2024-04-24T22:39:38.820936+0000","last_peered":"2024-04-24T22:39:38.820936+0000","last_clean":"2024-04-24T22:39:38.820936+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:39:38.820936+0000","last_undegraded":"2024-04-24T22:39:38.820936+0000","last_fullsized":"2024-04-24T22:39:38.820936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820780+0000","last_change":"2024-04-24T22:39:36.700237+0000","last_active":"2024-04-24T22:39:38.820780+0000","last_peered":"2024-04-24T22:39:38.820780+0000","last_clean":"2024-04-24T22:39:38.820780+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:39:38.820780+0000","last_undegraded":"2024-04-24T22:39:38.820780+0000","last_fullsized":"2024-04-24T22:39:38.820780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705984+0000","last_change":"2024-04-24T22:39:37.703532+0000","last_active":"2024-04-24T22:39:38.705984+0000","last_peered":"2024-04-24T22:39:38.705984+0000","last_clean":"2024-04-24T22:39:38.705984+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:39:38.705984+0000","last_undegraded":"2024-04-24T22:39:38.705984+0000","last_fullsized":"2024-04-24T22:39:38.705984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820432+0000","last_change":"2024-04-24T22:39:36.717477+0000","last_active":"2024-04-24T22:39:38.820432+0000","last_peered":"2024-04-24T22:39:38.820432+0000","last_clean":"2024-04-24T22:39:38.820432+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:39:38.820432+0000","last_undegraded":"2024-04-24T22:39:38.820432+0000","last_fullsized":"2024-04-24T22:39:38.820432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819989+0000","last_change":"2024-04-24T22:39:37.699255+0000","last_active":"2024-04-24T22:39:38.819989+0000","last_peered":"2024-04-24T22:39:38.819989+0000","last_clean":"2024-04-24T22:39:38.819989+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:39:38.819989+0000","last_undegraded":"2024-04-24T22:39:38.819989+0000","last_fullsized":"2024-04-24T22:39:38.819989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819412+0000","last_change":"2024-04-24T22:39:36.706567+0000","last_active":"2024-04-24T22:39:38.819412+0000","last_peered":"2024-04-24T22:39:38.819412+0000","last_clean":"2024-04-24T22:39:38.819412+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:39:38.819412+0000","last_undegraded":"2024-04-24T22:39:38.819412+0000","last_fullsized":"2024-04-24T22:39:38.819412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706118+0000","last_change":"2024-04-24T22:39:37.712064+0000","last_active":"2024-04-24T22:39:38.706118+0000","last_peered":"2024-04-24T22:39:38.706118+0000","last_clean":"2024-04-24T22:39:38.706118+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:39:38.706118+0000","last_undegraded":"2024-04-24T22:39:38.706118+0000","last_fullsized":"2024-04-24T22:39:38.706118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706281+0000","last_change":"2024-04-24T22:39:36.700041+0000","last_active":"2024-04-24T22:39:38.706281+0000","last_peered":"2024-04-24T22:39:38.706281+0000","last_clean":"2024-04-24T22:39:38.706281+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:39:38.706281+0000","last_undegraded":"2024-04-24T22:39:38.706281+0000","last_fullsized":"2024-04-24T22:39:38.706281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:50.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705269+0000","last_change":"2024-04-24T22:39:37.691459+0000","last_active":"2024-04-24T22:39:38.705269+0000","last_peered":"2024-04-24T22:39:38.705269+0000","last_clean":"2024-04-24T22:39:38.705269+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:39:38.705269+0000","last_undegraded":"2024-04-24T22:39:38.705269+0000","last_fullsized":"2024-04-24T22:39:38.705269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820416+0000","last_change":"2024-04-24T22:39:36.706454+0000","last_active":"2024-04-24T22:39:38.820416+0000","last_peered":"2024-04-24T22:39:38.820416+0000","last_clean":"2024-04-24T22:39:38.820416+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:39:38.820416+0000","last_undegraded":"2024-04-24T22:39:38.820416+0000","last_fullsized":"2024-04-24T22:39:38.820416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705898+0000","last_change":"2024-04-24T22:39:37.696724+0000","last_active":"2024-04-24T22:39:38.705898+0000","last_peered":"2024-04-24T22:39:38.705898+0000","last_clean":"2024-04-24T22:39:38.705898+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:39:38.705898+0000","last_undegraded":"2024-04-24T22:39:38.705898+0000","last_fullsized":"2024-04-24T22:39:38.705898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818924+0000","last_change":"2024-04-24T22:39:36.701340+0000","last_active":"2024-04-24T22:39:38.818924+0000","last_peered":"2024-04-24T22:39:38.818924+0000","last_clean":"2024-04-24T22:39:38.818924+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:39:38.818924+0000","last_undegraded":"2024-04-24T22:39:38.818924+0000","last_fullsized":"2024-04-24T22:39:38.818924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819009+0000","last_change":"2024-04-24T22:39:37.697532+0000","last_active":"2024-04-24T22:39:38.819009+0000","last_peered":"2024-04-24T22:39:38.819009+0000","last_clean":"2024-04-24T22:39:38.819009+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:39:38.819009+0000","last_undegraded":"2024-04-24T22:39:38.819009+0000","last_fullsized":"2024-04-24T22:39:38.819009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819879+0000","last_change":"2024-04-24T22:39:36.724649+0000","last_active":"2024-04-24T22:39:38.819879+0000","last_peered":"2024-04-24T22:39:38.819879+0000","last_clean":"2024-04-24T22:39:38.819879+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:39:38.819879+0000","last_undegraded":"2024-04-24T22:39:38.819879+0000","last_fullsized":"2024-04-24T22:39:38.819879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705839+0000","last_change":"2024-04-24T22:39:37.702890+0000","last_active":"2024-04-24T22:39:38.705839+0000","last_peered":"2024-04-24T22:39:38.705839+0000","last_clean":"2024-04-24T22:39:38.705839+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:39:38.705839+0000","last_undegraded":"2024-04-24T22:39:38.705839+0000","last_fullsized":"2024-04-24T22:39:38.705839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819608+0000","last_change":"2024-04-24T22:39:36.716044+0000","last_active":"2024-04-24T22:39:38.819608+0000","last_peered":"2024-04-24T22:39:38.819608+0000","last_clean":"2024-04-24T22:39:38.819608+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:39:38.819608+0000","last_undegraded":"2024-04-24T22:39:38.819608+0000","last_fullsized":"2024-04-24T22:39:38.819608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:40.023933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706077+0000","last_change":"2024-04-24T22:39:37.702621+0000","last_active":"2024-04-24T22:39:38.706077+0000","last_peered":"2024-04-24T22:39:38.706077+0000","last_clean":"2024-04-24T22:39:38.706077+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:39:38.706077+0000","last_undegraded":"2024-04-24T22:39:38.706077+0000","last_fullsized":"2024-04-24T22:39:38.706077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819737+0000","last_change":"2024-04-24T22:39:36.714272+0000","last_active":"2024-04-24T22:39:38.819737+0000","last_peered":"2024-04-24T22:39:38.819737+0000","last_clean":"2024-04-24T22:39:38.819737+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:39:38.819737+0000","last_undegraded":"2024-04-24T22:39:38.819737+0000","last_fullsized":"2024-04-24T22:39:38.819737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706373+0000","last_change":"2024-04-24T22:39:37.703786+0000","last_active":"2024-04-24T22:39:38.706373+0000","last_peered":"2024-04-24T22:39:38.706373+0000","last_clean":"2024-04-24T22:39:38.706373+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:39:38.706373+0000","last_undegraded":"2024-04-24T22:39:38.706373+0000","last_fullsized":"2024-04-24T22:39:38.706373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:55.990656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820545+0000","last_change":"2024-04-24T22:39:36.715437+0000","last_active":"2024-04-24T22:39:38.820545+0000","last_peered":"2024-04-24T22:39:38.820545+0000","last_clean":"2024-04-24T22:39:38.820545+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:39:38.820545+0000","last_undegraded":"2024-04-24T22:39:38.820545+0000","last_fullsized":"2024-04-24T22:39:38.820545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820735+0000","last_change":"2024-04-24T22:39:37.715801+0000","last_active":"2024-04-24T22:39:38.820735+0000","last_peered":"2024-04-24T22:39:38.820735+0000","last_clean":"2024-04-24T22:39:38.820735+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:39:38.820735+0000","last_undegraded":"2024-04-24T22:39:38.820735+0000","last_fullsized":"2024-04-24T22:39:38.820735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819581+0000","last_change":"2024-04-24T22:39:36.710422+0000","last_active":"2024-04-24T22:39:38.819581+0000","last_peered":"2024-04-24T22:39:38.819581+0000","last_clean":"2024-04-24T22:39:38.819581+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:39:38.819581+0000","last_undegraded":"2024-04-24T22:39:38.819581+0000","last_fullsized":"2024-04-24T22:39:38.819581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706031+0000","last_change":"2024-04-24T22:39:37.706745+0000","last_active":"2024-04-24T22:39:38.706031+0000","last_peered":"2024-04-24T22:39:38.706031+0000","last_clean":"2024-04-24T22:39:38.706031+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:39:38.706031+0000","last_undegraded":"2024-04-24T22:39:38.706031+0000","last_fullsized":"2024-04-24T22:39:38.706031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705671+0000","last_change":"2024-04-24T22:39:32.657965+0000","last_active":"2024-04-24T22:39:38.705671+0000","last_peered":"2024-04-24T22:39:38.705671+0000","last_clean":"2024-04-24T22:39:38.705671+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:39:38.705671+0000","last_undegraded":"2024-04-24T22:39:38.705671+0000","last_fullsized":"2024-04-24T22:39:38.705671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_clean_scrub_stamp":"2024-04-24T22:39:31.635804+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.714153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820290+0000","last_change":"2024-04-24T22:39:36.710381+0000","last_active":"2024-04-24T22:39:38.820290+0000","last_peered":"2024-04-24T22:39:38.820290+0000","last_clean":"2024-04-24T22:39:38.820290+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:39:38.820290+0000","last_undegraded":"2024-04-24T22:39:38.820290+0000","last_fullsized":"2024-04-24T22:39:38.820290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708707+0000","last_change":"2024-04-24T22:39:37.713956+0000","last_active":"2024-04-24T22:39:38.708707+0000","last_peered":"2024-04-24T22:39:38.708707+0000","last_clean":"2024-04-24T22:39:38.708707+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:39:38.708707+0000","last_undegraded":"2024-04-24T22:39:38.708707+0000","last_fullsized":"2024-04-24T22:39:38.708707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705533+0000","last_change":"2024-04-24T22:39:36.715290+0000","last_active":"2024-04-24T22:39:38.705533+0000","last_peered":"2024-04-24T22:39:38.705533+0000","last_clean":"2024-04-24T22:39:38.705533+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:39:38.705533+0000","last_undegraded":"2024-04-24T22:39:38.705533+0000","last_fullsized":"2024-04-24T22:39:38.705533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704538+0000","last_change":"2024-04-24T22:39:37.695419+0000","last_active":"2024-04-24T22:39:38.704538+0000","last_peered":"2024-04-24T22:39:38.704538+0000","last_clean":"2024-04-24T22:39:38.704538+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:39:38.704538+0000","last_undegraded":"2024-04-24T22:39:38.704538+0000","last_fullsized":"2024-04-24T22:39:38.704538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:08.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819608+0000","last_change":"2024-04-24T22:39:36.707548+0000","last_active":"2024-04-24T22:39:38.819608+0000","last_peered":"2024-04-24T22:39:38.819608+0000","last_clean":"2024-04-24T22:39:38.819608+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:39:38.819608+0000","last_undegraded":"2024-04-24T22:39:38.819608+0000","last_fullsized":"2024-04-24T22:39:38.819608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820375+0000","last_change":"2024-04-24T22:39:37.703301+0000","last_active":"2024-04-24T22:39:38.820375+0000","last_peered":"2024-04-24T22:39:38.820375+0000","last_clean":"2024-04-24T22:39:38.820375+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:39:38.820375+0000","last_undegraded":"2024-04-24T22:39:38.820375+0000","last_fullsized":"2024-04-24T22:39:38.820375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820564+0000","last_change":"2024-04-24T22:39:36.713486+0000","last_active":"2024-04-24T22:39:38.820564+0000","last_peered":"2024-04-24T22:39:38.820564+0000","last_clean":"2024-04-24T22:39:38.820564+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:39:38.820564+0000","last_undegraded":"2024-04-24T22:39:38.820564+0000","last_fullsized":"2024-04-24T22:39:38.820564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819526+0000","last_change":"2024-04-24T22:39:37.705705+0000","last_active":"2024-04-24T22:39:38.819526+0000","last_peered":"2024-04-24T22:39:38.819526+0000","last_clean":"2024-04-24T22:39:38.819526+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:39:38.819526+0000","last_undegraded":"2024-04-24T22:39:38.819526+0000","last_fullsized":"2024-04-24T22:39:38.819526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:34.424877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820670+0000","last_change":"2024-04-24T22:39:36.723125+0000","last_active":"2024-04-24T22:39:38.820670+0000","last_peered":"2024-04-24T22:39:38.820670+0000","last_clean":"2024-04-24T22:39:38.820670+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:39:38.820670+0000","last_undegraded":"2024-04-24T22:39:38.820670+0000","last_fullsized":"2024-04-24T22:39:38.820670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820922+0000","last_change":"2024-04-24T22:39:37.713947+0000","last_active":"2024-04-24T22:39:38.820922+0000","last_peered":"2024-04-24T22:39:38.820922+0000","last_clean":"2024-04-24T22:39:38.820922+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:39:38.820922+0000","last_undegraded":"2024-04-24T22:39:38.820922+0000","last_fullsized":"2024-04-24T22:39:38.820922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706225+0000","last_change":"2024-04-24T22:39:36.716541+0000","last_active":"2024-04-24T22:39:38.706225+0000","last_peered":"2024-04-24T22:39:38.706225+0000","last_clean":"2024-04-24T22:39:38.706225+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:39:38.706225+0000","last_undegraded":"2024-04-24T22:39:38.706225+0000","last_fullsized":"2024-04-24T22:39:38.706225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:33.366773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820580+0000","last_change":"2024-04-24T22:39:37.708069+0000","last_active":"2024-04-24T22:39:38.820580+0000","last_peered":"2024-04-24T22:39:38.820580+0000","last_clean":"2024-04-24T22:39:38.820580+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:39:38.820580+0000","last_undegraded":"2024-04-24T22:39:38.820580+0000","last_fullsized":"2024-04-24T22:39:38.820580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.821341+0000","last_change":"2024-04-24T22:39:36.718740+0000","last_active":"2024-04-24T22:39:38.821341+0000","last_peered":"2024-04-24T22:39:38.821341+0000","last_clean":"2024-04-24T22:39:38.821341+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:39:38.821341+0000","last_undegraded":"2024-04-24T22:39:38.821341+0000","last_fullsized":"2024-04-24T22:39:38.821341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.709326+0000","last_change":"2024-04-24T22:39:37.704496+0000","last_active":"2024-04-24T22:39:38.709326+0000","last_peered":"2024-04-24T22:39:38.709326+0000","last_clean":"2024-04-24T22:39:38.709326+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:39:38.709326+0000","last_undegraded":"2024-04-24T22:39:38.709326+0000","last_fullsized":"2024-04-24T22:39:38.709326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819467+0000","last_change":"2024-04-24T22:39:36.701308+0000","last_active":"2024-04-24T22:39:38.819467+0000","last_peered":"2024-04-24T22:39:38.819467+0000","last_clean":"2024-04-24T22:39:38.819467+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:39:38.819467+0000","last_undegraded":"2024-04-24T22:39:38.819467+0000","last_fullsized":"2024-04-24T22:39:38.819467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705483+0000","last_change":"2024-04-24T22:39:37.700189+0000","last_active":"2024-04-24T22:39:38.705483+0000","last_peered":"2024-04-24T22:39:38.705483+0000","last_clean":"2024-04-24T22:39:38.705483+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:39:38.705483+0000","last_undegraded":"2024-04-24T22:39:38.705483+0000","last_fullsized":"2024-04-24T22:39:38.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705533+0000","last_change":"2024-04-24T22:39:36.716460+0000","last_active":"2024-04-24T22:39:38.705533+0000","last_peered":"2024-04-24T22:39:38.705533+0000","last_clean":"2024-04-24T22:39:38.705533+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:39:38.705533+0000","last_undegraded":"2024-04-24T22:39:38.705533+0000","last_fullsized":"2024-04-24T22:39:38.705533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819020+0000","last_change":"2024-04-24T22:39:37.704143+0000","last_active":"2024-04-24T22:39:38.819020+0000","last_peered":"2024-04-24T22:39:38.819020+0000","last_clean":"2024-04-24T22:39:38.819020+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:39:38.819020+0000","last_undegraded":"2024-04-24T22:39:38.819020+0000","last_fullsized":"2024-04-24T22:39:38.819020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:58.556519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820279+0000","last_change":"2024-04-24T22:39:36.721540+0000","last_active":"2024-04-24T22:39:38.820279+0000","last_peered":"2024-04-24T22:39:38.820279+0000","last_clean":"2024-04-24T22:39:38.820279+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:39:38.820279+0000","last_undegraded":"2024-04-24T22:39:38.820279+0000","last_fullsized":"2024-04-24T22:39:38.820279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708008+0000","last_change":"2024-04-24T22:39:37.714150+0000","last_active":"2024-04-24T22:39:38.708008+0000","last_peered":"2024-04-24T22:39:38.708008+0000","last_clean":"2024-04-24T22:39:38.708008+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:39:38.708008+0000","last_undegraded":"2024-04-24T22:39:38.708008+0000","last_fullsized":"2024-04-24T22:39:38.708008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820479+0000","last_change":"2024-04-24T22:39:36.716667+0000","last_active":"2024-04-24T22:39:38.820479+0000","last_peered":"2024-04-24T22:39:38.820479+0000","last_clean":"2024-04-24T22:39:38.820479+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:39:38.820479+0000","last_undegraded":"2024-04-24T22:39:38.820479+0000","last_fullsized":"2024-04-24T22:39:38.820479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708586+0000","last_change":"2024-04-24T22:39:37.715906+0000","last_active":"2024-04-24T22:39:38.708586+0000","last_peered":"2024-04-24T22:39:38.708586+0000","last_clean":"2024-04-24T22:39:38.708586+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:39:38.708586+0000","last_undegraded":"2024-04-24T22:39:38.708586+0000","last_fullsized":"2024-04-24T22:39:38.708586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706034+0000","last_change":"2024-04-24T22:39:36.698212+0000","last_active":"2024-04-24T22:39:38.706034+0000","last_peered":"2024-04-24T22:39:38.706034+0000","last_clean":"2024-04-24T22:39:38.706034+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:39:38.706034+0000","last_undegraded":"2024-04-24T22:39:38.706034+0000","last_fullsized":"2024-04-24T22:39:38.706034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818963+0000","last_change":"2024-04-24T22:39:37.696143+0000","last_active":"2024-04-24T22:39:38.818963+0000","last_peered":"2024-04-24T22:39:38.818963+0000","last_clean":"2024-04-24T22:39:38.818963+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:39:38.818963+0000","last_undegraded":"2024-04-24T22:39:38.818963+0000","last_fullsized":"2024-04-24T22:39:38.818963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820834+0000","last_change":"2024-04-24T22:39:37.696113+0000","last_active":"2024-04-24T22:39:38.820834+0000","last_peered":"2024-04-24T22:39:38.820834+0000","last_clean":"2024-04-24T22:39:38.820834+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:39:38.820834+0000","last_undegraded":"2024-04-24T22:39:38.820834+0000","last_fullsized":"2024-04-24T22:39:38.820834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705916+0000","last_change":"2024-04-24T22:39:36.699149+0000","last_active":"2024-04-24T22:39:38.705916+0000","last_peered":"2024-04-24T22:39:38.705916+0000","last_clean":"2024-04-24T22:39:38.705916+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:39:38.705916+0000","last_undegraded":"2024-04-24T22:39:38.705916+0000","last_fullsized":"2024-04-24T22:39:38.705916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820550+0000","last_change":"2024-04-24T22:39:37.700972+0000","last_active":"2024-04-24T22:39:38.820550+0000","last_peered":"2024-04-24T22:39:38.820550+0000","last_clean":"2024-04-24T22:39:38.820550+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:39:38.820550+0000","last_undegraded":"2024-04-24T22:39:38.820550+0000","last_fullsized":"2024-04-24T22:39:38.820550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706124+0000","last_change":"2024-04-24T22:39:36.698567+0000","last_active":"2024-04-24T22:39:38.706124+0000","last_peered":"2024-04-24T22:39:38.706124+0000","last_clean":"2024-04-24T22:39:38.706124+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:39:38.706124+0000","last_undegraded":"2024-04-24T22:39:38.706124+0000","last_fullsized":"2024-04-24T22:39:38.706124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706000+0000","last_change":"2024-04-24T22:39:37.706951+0000","last_active":"2024-04-24T22:39:38.706000+0000","last_peered":"2024-04-24T22:39:38.706000+0000","last_clean":"2024-04-24T22:39:38.706000+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:39:38.706000+0000","last_undegraded":"2024-04-24T22:39:38.706000+0000","last_fullsized":"2024-04-24T22:39:38.706000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708849+0000","last_change":"2024-04-24T22:39:36.703823+0000","last_active":"2024-04-24T22:39:38.708849+0000","last_peered":"2024-04-24T22:39:38.708849+0000","last_clean":"2024-04-24T22:39:38.708849+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:39:38.708849+0000","last_undegraded":"2024-04-24T22:39:38.708849+0000","last_fullsized":"2024-04-24T22:39:38.708849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708601+0000","last_change":"2024-04-24T22:39:37.712078+0000","last_active":"2024-04-24T22:39:38.708601+0000","last_peered":"2024-04-24T22:39:38.708601+0000","last_clean":"2024-04-24T22:39:38.708601+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:39:38.708601+0000","last_undegraded":"2024-04-24T22:39:38.708601+0000","last_fullsized":"2024-04-24T22:39:38.708601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706267+0000","last_change":"2024-04-24T22:39:36.719643+0000","last_active":"2024-04-24T22:39:38.706267+0000","last_peered":"2024-04-24T22:39:38.706267+0000","last_clean":"2024-04-24T22:39:38.706267+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:39:38.706267+0000","last_undegraded":"2024-04-24T22:39:38.706267+0000","last_fullsized":"2024-04-24T22:39:38.706267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":665973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":665973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:39:44.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:39:44.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.412+0000 7f4911a00640 1 Processor -- start 2024-04-24T22:39:44.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.412+0000 7f4911a00640 1 -- start start 2024-04-24T22:39:44.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.414+0000 7f4911a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f490c073280 0x7f490c0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:44.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.414+0000 7f4911a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f490c15b2c0 con 0x7f490c154560 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.414+0000 7f4911a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f490c15da90 con 0x7f490c0cbb40 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.414+0000 7f4911a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f490c15dd70 con 0x7f490c073280 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.414+0000 7f490be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f490c154560 0x7f490c0cb430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54440/0 (socket says 172.21.15.47:54440) 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.414+0000 7f490be00640 1 -- 172.21.15.47:0/1416225725 learned_addr learned my addr 172.21.15.47:0/1416225725 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.414+0000 7f4911000640 1 --2- 172.21.15.47:0/1416225725 >> v2:172.21.15.113:3300/0 conn(0x7f490c073280 0x7f490c0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.415+0000 7f490b400640 1 -- 172.21.15.47:0/1416225725 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4090684224 0 0) 0x7f490c15b2c0 con 0x7f490c154560 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.415+0000 7f490b400640 1 -- 172.21.15.47:0/1416225725 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48f0003520 con 0x7f490c154560 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.415+0000 7f490b400640 1 -- 172.21.15.47:0/1416225725 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003332609 0 0) 0x7f490c15da90 con 0x7f490c0cbb40 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.415+0000 7f490b400640 1 -- 172.21.15.47:0/1416225725 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f490c15b2c0 con 0x7f490c0cbb40 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.415+0000 7f4911000640 1 -- 172.21.15.47:0/1416225725 >> v1:172.21.15.113:6789/0 conn(0x7f490c0cbb40 legacy=0x7f490c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.415+0000 7f4911000640 1 -- 172.21.15.47:0/1416225725 >> v1:172.21.15.47:6789/0 conn(0x7f490c154560 legacy=0x7f490c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.415+0000 7f4911000640 1 -- 172.21.15.47:0/1416225725 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c15e050 con 0x7f490c073280 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.416+0000 7f4911000640 1 --2- 172.21.15.47:0/1416225725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490c073280 0x7f490c0ccd70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4904009d90 tx=0x7f490402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa029b507a6070f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.416+0000 7f490b400640 1 -- 172.21.15.47:0/1416225725 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904007740 con 0x7f490c073280 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.416+0000 7f490b400640 1 -- 172.21.15.47:0/1416225725 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49040078e0 con 0x7f490c073280 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.416+0000 7f490b400640 1 -- 172.21.15.47:0/1416225725 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904007cb0 con 0x7f490c073280 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.417+0000 7f4911a00640 1 -- 172.21.15.47:0/1416225725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490c073280 msgr2=0x7f490c0ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.417+0000 7f4911a00640 1 --2- 172.21.15.47:0/1416225725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490c073280 0x7f490c0ccd70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4904009d90 tx=0x7f490402a660 comp rx=0 tx=0).stop 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.417+0000 7f4911a00640 1 -- 172.21.15.47:0/1416225725 shutdown_connections 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.417+0000 7f4911a00640 1 --2- 172.21.15.47:0/1416225725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490c073280 0x7f490c0ccd70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.417+0000 7f4911a00640 1 -- 172.21.15.47:0/1416225725 >> 172.21.15.47:0/1416225725 conn(0x7f490c0672c0 msgr2=0x7f490c072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.417+0000 7f4911a00640 1 -- 172.21.15.47:0/1416225725 shutdown_connections 2024-04-24T22:39:44.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.417+0000 7f4911a00640 1 -- 172.21.15.47:0/1416225725 wait complete. 2024-04-24T22:39:44.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.419+0000 7f4911a00640 1 Processor -- start 2024-04-24T22:39:44.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.419+0000 7f4911a00640 1 -- start start 2024-04-24T22:39:44.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f4911a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f490c073280 0x7f490c0b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:44.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f4911a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490c0cbb40 0x7f490c148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:44.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f4911a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f490c154560 0x7f490c148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:44.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f4911a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f490c0cd2b0 con 0x7f490c073280 2024-04-24T22:39:44.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f4911a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f490c0caad0 con 0x7f490c0cbb40 2024-04-24T22:39:44.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f4911a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f490c15ae80 con 0x7f490c154560 2024-04-24T22:39:44.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f490be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f490c073280 0x7f490c0b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f490be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f490c073280 0x7f490c0b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53662/0 (socket says 172.21.15.47:53662) 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f490be00640 1 -- 172.21.15.47:0/3138532144 learned_addr learned my addr 172.21.15.47:0/3138532144 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f4903e00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490c0cbb40 0x7f490c148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.420+0000 7f4911000640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f490c154560 0x7f490c148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.421+0000 7f490be00640 1 -- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f490c154560 msgr2=0x7f490c148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.421+0000 7f490be00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f490c154560 0x7f490c148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.421+0000 7f490be00640 1 -- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490c0cbb40 msgr2=0x7f490c148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.421+0000 7f490be00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490c0cbb40 0x7f490c148570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.421+0000 7f490be00640 1 -- 172.21.15.47:0/3138532144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c1f0c60 con 0x7f490c073280 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.421+0000 7f490be00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f490c073280 0x7f490c0b4a20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00c530 tx=0x7f48fc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:44.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.421+0000 7f4908c00640 1 -- 172.21.15.47:0/3138532144 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc017070 con 0x7f490c073280 2024-04-24T22:39:44.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.421+0000 7f4908c00640 1 -- 172.21.15.47:0/3138532144 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48fc005ce0 con 0x7f490c073280 2024-04-24T22:39:44.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.421+0000 7f4908c00640 1 -- 172.21.15.47:0/3138532144 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc007270 con 0x7f490c073280 2024-04-24T22:39:44.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.421+0000 7f4911a00640 1 -- 172.21.15.47:0/3138532144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c1f0f40 con 0x7f490c073280 2024-04-24T22:39:44.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.422+0000 7f4911a00640 1 -- 172.21.15.47:0/3138532144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1f1500 con 0x7f490c073280 2024-04-24T22:39:44.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.423+0000 7f4911a00640 1 -- 172.21.15.47:0/3138532144 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490c0c6340 con 0x7f490c073280 2024-04-24T22:39:44.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.424+0000 7f4908c00640 1 -- 172.21.15.47:0/3138532144 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48fc01a070 con 0x7f490c073280 2024-04-24T22:39:44.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.425+0000 7f4908c00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f48e806be30 0x7f48e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:44.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.425+0000 7f4908c00640 1 -- 172.21.15.47:0/3138532144 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f48fc08a280 con 0x7f490c073280 2024-04-24T22:39:44.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.425+0000 7f4903e00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f48e806be30 0x7f48e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:44.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.426+0000 7f4903e00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f48e806be30 0x7f48e806e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f48f4007fd0 tx=0x7f48f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:44.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.429+0000 7f4908c00640 1 -- 172.21.15.47:0/3138532144 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f490c0c6340 con 0x7f490c073280 2024-04-24T22:39:44.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.630+0000 7f4911a00640 1 -- 172.21.15.47:0/3138532144 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f490c0cf970 con 0x7f48e806be30 2024-04-24T22:39:44.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.637+0000 7f4908c00640 1 -- 172.21.15.47:0/3138532144 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251845 (secure 0 0 0) 0x7f490c0cf970 con 0x7f48e806be30 2024-04-24T22:39:44.643 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:39:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.643+0000 7f4911a00640 1 -- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f48e806be30 msgr2=0x7f48e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.643+0000 7f4911a00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f48e806be30 0x7f48e806e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f48f4007fd0 tx=0x7f48f4009040 comp rx=0 tx=0).stop 2024-04-24T22:39:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.643+0000 7f4911a00640 1 -- 172.21.15.47:0/3138532144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f490c073280 msgr2=0x7f490c0b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.643+0000 7f4911a00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f490c073280 0x7f490c0b4a20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00c530 tx=0x7f48fc00c9f0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.643+0000 7f4911a00640 1 -- 172.21.15.47:0/3138532144 shutdown_connections 2024-04-24T22:39:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.643+0000 7f4911a00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f48e806be30 0x7f48e806e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.643+0000 7f4911a00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f490c154560 0x7f490c148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.643+0000 7f4911a00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490c0cbb40 0x7f490c148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.643+0000 7f4911a00640 1 --2- 172.21.15.47:0/3138532144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f490c073280 0x7f490c0b4a20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.643+0000 7f4911a00640 1 -- 172.21.15.47:0/3138532144 >> 172.21.15.47:0/3138532144 conn(0x7f490c0672c0 msgr2=0x7f490c0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:44.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.644+0000 7f4911a00640 1 -- 172.21.15.47:0/3138532144 shutdown_connections 2024-04-24T22:39:44.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.644+0000 7f4911a00640 1 -- 172.21.15.47:0/3138532144 wait complete. 2024-04-24T22:39:44.668 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-24T22:39:42.642959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235708,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739012,"statfs":{"total":773094113280,"available":772852748288,"internally_reserved":0,"allocated":2924544,"data_stored":1785576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001379"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706276+0000","last_change":"2024-04-24T22:39:37.699192+0000","last_active":"2024-04-24T22:39:38.706276+0000","last_peered":"2024-04-24T22:39:38.706276+0000","last_clean":"2024-04-24T22:39:38.706276+0000","last_became_active":"2024-04-24T22:39:37.698920+0000","last_became_peered":"2024-04-24T22:39:37.698920+0000","last_unstale":"2024-04-24T22:39:38.706276+0000","last_undegraded":"2024-04-24T22:39:38.706276+0000","last_fullsized":"2024-04-24T22:39:38.706276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.821236+0000","last_change":"2024-04-24T22:39:37.697245+0000","last_active":"2024-04-24T22:39:38.821236+0000","last_peered":"2024-04-24T22:39:38.821236+0000","last_clean":"2024-04-24T22:39:38.821236+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:39:38.821236+0000","last_undegraded":"2024-04-24T22:39:38.821236+0000","last_fullsized":"2024-04-24T22:39:38.821236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705212+0000","last_change":"2024-04-24T22:39:37.709539+0000","last_active":"2024-04-24T22:39:38.705212+0000","last_peered":"2024-04-24T22:39:38.705212+0000","last_clean":"2024-04-24T22:39:38.705212+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:39:38.705212+0000","last_undegraded":"2024-04-24T22:39:38.705212+0000","last_fullsized":"2024-04-24T22:39:38.705212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820523+0000","last_change":"2024-04-24T22:39:37.719136+0000","last_active":"2024-04-24T22:39:38.820523+0000","last_peered":"2024-04-24T22:39:38.820523+0000","last_clean":"2024-04-24T22:39:38.820523+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:39:38.820523+0000","last_undegraded":"2024-04-24T22:39:38.820523+0000","last_fullsized":"2024-04-24T22:39:38.820523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705940+0000","last_change":"2024-04-24T22:39:37.704671+0000","last_active":"2024-04-24T22:39:38.705940+0000","last_peered":"2024-04-24T22:39:38.705940+0000","last_clean":"2024-04-24T22:39:38.705940+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:39:38.705940+0000","last_undegraded":"2024-04-24T22:39:38.705940+0000","last_fullsized":"2024-04-24T22:39:38.705940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820279+0000","last_change":"2024-04-24T22:39:37.703880+0000","last_active":"2024-04-24T22:39:38.820279+0000","last_peered":"2024-04-24T22:39:38.820279+0000","last_clean":"2024-04-24T22:39:38.820279+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:39:38.820279+0000","last_undegraded":"2024-04-24T22:39:38.820279+0000","last_fullsized":"2024-04-24T22:39:38.820279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705748+0000","last_change":"2024-04-24T22:39:37.703539+0000","last_active":"2024-04-24T22:39:38.705748+0000","last_peered":"2024-04-24T22:39:38.705748+0000","last_clean":"2024-04-24T22:39:38.705748+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:39:38.705748+0000","last_undegraded":"2024-04-24T22:39:38.705748+0000","last_fullsized":"2024-04-24T22:39:38.705748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707950+0000","last_change":"2024-04-24T22:39:37.713543+0000","last_active":"2024-04-24T22:39:38.707950+0000","last_peered":"2024-04-24T22:39:38.707950+0000","last_clean":"2024-04-24T22:39:38.707950+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:39:38.707950+0000","last_undegraded":"2024-04-24T22:39:38.707950+0000","last_fullsized":"2024-04-24T22:39:38.707950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706353+0000","last_change":"2024-04-24T22:39:37.706715+0000","last_active":"2024-04-24T22:39:38.706353+0000","last_peered":"2024-04-24T22:39:38.706353+0000","last_clean":"2024-04-24T22:39:38.706353+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:39:38.706353+0000","last_undegraded":"2024-04-24T22:39:38.706353+0000","last_fullsized":"2024-04-24T22:39:38.706353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819558+0000","last_change":"2024-04-24T22:39:37.701250+0000","last_active":"2024-04-24T22:39:38.819558+0000","last_peered":"2024-04-24T22:39:38.819558+0000","last_clean":"2024-04-24T22:39:38.819558+0000","last_became_active":"2024-04-24T22:39:37.700959+0000","last_became_peered":"2024-04-24T22:39:37.700959+0000","last_unstale":"2024-04-24T22:39:38.819558+0000","last_undegraded":"2024-04-24T22:39:38.819558+0000","last_fullsized":"2024-04-24T22:39:38.819558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819186+0000","last_change":"2024-04-24T22:39:37.701069+0000","last_active":"2024-04-24T22:39:38.819186+0000","last_peered":"2024-04-24T22:39:38.819186+0000","last_clean":"2024-04-24T22:39:38.819186+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:39:38.819186+0000","last_undegraded":"2024-04-24T22:39:38.819186+0000","last_fullsized":"2024-04-24T22:39:38.819186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819923+0000","last_change":"2024-04-24T22:39:37.693508+0000","last_active":"2024-04-24T22:39:38.819923+0000","last_peered":"2024-04-24T22:39:38.819923+0000","last_clean":"2024-04-24T22:39:38.819923+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:39:38.819923+0000","last_undegraded":"2024-04-24T22:39:38.819923+0000","last_fullsized":"2024-04-24T22:39:38.819923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708270+0000","last_change":"2024-04-24T22:39:37.705157+0000","last_active":"2024-04-24T22:39:38.708270+0000","last_peered":"2024-04-24T22:39:38.708270+0000","last_clean":"2024-04-24T22:39:38.708270+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:39:38.708270+0000","last_undegraded":"2024-04-24T22:39:38.708270+0000","last_fullsized":"2024-04-24T22:39:38.708270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704755+0000","last_change":"2024-04-24T22:39:37.698333+0000","last_active":"2024-04-24T22:39:38.704755+0000","last_peered":"2024-04-24T22:39:38.704755+0000","last_clean":"2024-04-24T22:39:38.704755+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:39:38.704755+0000","last_undegraded":"2024-04-24T22:39:38.704755+0000","last_fullsized":"2024-04-24T22:39:38.704755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:57.510197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820219+0000","last_change":"2024-04-24T22:39:37.696874+0000","last_active":"2024-04-24T22:39:38.820219+0000","last_peered":"2024-04-24T22:39:38.820219+0000","last_clean":"2024-04-24T22:39:38.820219+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:39:38.820219+0000","last_undegraded":"2024-04-24T22:39:38.820219+0000","last_fullsized":"2024-04-24T22:39:38.820219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820021+0000","last_change":"2024-04-24T22:39:37.703446+0000","last_active":"2024-04-24T22:39:38.820021+0000","last_peered":"2024-04-24T22:39:38.820021+0000","last_clean":"2024-04-24T22:39:38.820021+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:39:38.820021+0000","last_undegraded":"2024-04-24T22:39:38.820021+0000","last_fullsized":"2024-04-24T22:39:38.820021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820971+0000","last_change":"2024-04-24T22:39:37.701515+0000","last_active":"2024-04-24T22:39:38.820971+0000","last_peered":"2024-04-24T22:39:38.820971+0000","last_clean":"2024-04-24T22:39:38.820971+0000","last_became_active":"2024-04-24T22:39:37.700439+0000","last_became_peered":"2024-04-24T22:39:37.700439+0000","last_unstale":"2024-04-24T22:39:38.820971+0000","last_undegraded":"2024-04-24T22:39:38.820971+0000","last_fullsized":"2024-04-24T22:39:38.820971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819628+0000","last_change":"2024-04-24T22:39:37.703899+0000","last_active":"2024-04-24T22:39:38.819628+0000","last_peered":"2024-04-24T22:39:38.819628+0000","last_clean":"2024-04-24T22:39:38.819628+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:39:38.819628+0000","last_undegraded":"2024-04-24T22:39:38.819628+0000","last_fullsized":"2024-04-24T22:39:38.819628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706244+0000","last_change":"2024-04-24T22:39:37.711019+0000","last_active":"2024-04-24T22:39:38.706244+0000","last_peered":"2024-04-24T22:39:38.706244+0000","last_clean":"2024-04-24T22:39:38.706244+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:39:38.706244+0000","last_undegraded":"2024-04-24T22:39:38.706244+0000","last_fullsized":"2024-04-24T22:39:38.706244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819357+0000","last_change":"2024-04-24T22:39:37.696214+0000","last_active":"2024-04-24T22:39:38.819357+0000","last_peered":"2024-04-24T22:39:38.819357+0000","last_clean":"2024-04-24T22:39:38.819357+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:39:38.819357+0000","last_undegraded":"2024-04-24T22:39:38.819357+0000","last_fullsized":"2024-04-24T22:39:38.819357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705029+0000","last_change":"2024-04-24T22:39:37.700151+0000","last_active":"2024-04-24T22:39:38.705029+0000","last_peered":"2024-04-24T22:39:38.705029+0000","last_clean":"2024-04-24T22:39:38.705029+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:39:38.705029+0000","last_undegraded":"2024-04-24T22:39:38.705029+0000","last_fullsized":"2024-04-24T22:39:38.705029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820267+0000","last_change":"2024-04-24T22:39:37.705083+0000","last_active":"2024-04-24T22:39:38.820267+0000","last_peered":"2024-04-24T22:39:38.820267+0000","last_clean":"2024-04-24T22:39:38.820267+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:39:38.820267+0000","last_undegraded":"2024-04-24T22:39:38.820267+0000","last_fullsized":"2024-04-24T22:39:38.820267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708194+0000","last_change":"2024-04-24T22:39:37.711193+0000","last_active":"2024-04-24T22:39:38.708194+0000","last_peered":"2024-04-24T22:39:38.708194+0000","last_clean":"2024-04-24T22:39:38.708194+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:39:38.708194+0000","last_undegraded":"2024-04-24T22:39:38.708194+0000","last_fullsized":"2024-04-24T22:39:38.708194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820128+0000","last_change":"2024-04-24T22:39:37.713910+0000","last_active":"2024-04-24T22:39:38.820128+0000","last_peered":"2024-04-24T22:39:38.820128+0000","last_clean":"2024-04-24T22:39:38.820128+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:39:38.820128+0000","last_undegraded":"2024-04-24T22:39:38.820128+0000","last_fullsized":"2024-04-24T22:39:38.820128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708266+0000","last_change":"2024-04-24T22:39:37.717191+0000","last_active":"2024-04-24T22:39:38.708266+0000","last_peered":"2024-04-24T22:39:38.708266+0000","last_clean":"2024-04-24T22:39:38.708266+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:39:38.708266+0000","last_undegraded":"2024-04-24T22:39:38.708266+0000","last_fullsized":"2024-04-24T22:39:38.708266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819555+0000","last_change":"2024-04-24T22:39:37.702983+0000","last_active":"2024-04-24T22:39:38.819555+0000","last_peered":"2024-04-24T22:39:38.819555+0000","last_clean":"2024-04-24T22:39:38.819555+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:39:38.819555+0000","last_undegraded":"2024-04-24T22:39:38.819555+0000","last_fullsized":"2024-04-24T22:39:38.819555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820583+0000","last_change":"2024-04-24T22:39:37.714111+0000","last_active":"2024-04-24T22:39:38.820583+0000","last_peered":"2024-04-24T22:39:38.820583+0000","last_clean":"2024-04-24T22:39:38.820583+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:39:38.820583+0000","last_undegraded":"2024-04-24T22:39:38.820583+0000","last_fullsized":"2024-04-24T22:39:38.820583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708923+0000","last_change":"2024-04-24T22:39:37.714877+0000","last_active":"2024-04-24T22:39:38.708923+0000","last_peered":"2024-04-24T22:39:38.708923+0000","last_clean":"2024-04-24T22:39:38.708923+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:39:38.708923+0000","last_undegraded":"2024-04-24T22:39:38.708923+0000","last_fullsized":"2024-04-24T22:39:38.708923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705236+0000","last_change":"2024-04-24T22:39:37.714695+0000","last_active":"2024-04-24T22:39:38.705236+0000","last_peered":"2024-04-24T22:39:38.705236+0000","last_clean":"2024-04-24T22:39:38.705236+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:39:38.705236+0000","last_undegraded":"2024-04-24T22:39:38.705236+0000","last_fullsized":"2024-04-24T22:39:38.705236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820461+0000","last_change":"2024-04-24T22:39:37.719187+0000","last_active":"2024-04-24T22:39:38.820461+0000","last_peered":"2024-04-24T22:39:38.820461+0000","last_clean":"2024-04-24T22:39:38.820461+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:39:38.820461+0000","last_undegraded":"2024-04-24T22:39:38.820461+0000","last_fullsized":"2024-04-24T22:39:38.820461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819723+0000","last_change":"2024-04-24T22:39:37.707283+0000","last_active":"2024-04-24T22:39:38.819723+0000","last_peered":"2024-04-24T22:39:38.819723+0000","last_clean":"2024-04-24T22:39:38.819723+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:39:38.819723+0000","last_undegraded":"2024-04-24T22:39:38.819723+0000","last_fullsized":"2024-04-24T22:39:38.819723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818968+0000","last_change":"2024-04-24T22:39:37.697902+0000","last_active":"2024-04-24T22:39:38.818968+0000","last_peered":"2024-04-24T22:39:38.818968+0000","last_clean":"2024-04-24T22:39:38.818968+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:39:38.818968+0000","last_undegraded":"2024-04-24T22:39:38.818968+0000","last_fullsized":"2024-04-24T22:39:38.818968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819142+0000","last_change":"2024-04-24T22:39:37.705667+0000","last_active":"2024-04-24T22:39:38.819142+0000","last_peered":"2024-04-24T22:39:38.819142+0000","last_clean":"2024-04-24T22:39:38.819142+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:39:38.819142+0000","last_undegraded":"2024-04-24T22:39:38.819142+0000","last_fullsized":"2024-04-24T22:39:38.819142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:50.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708783+0000","last_change":"2024-04-24T22:39:36.707990+0000","last_active":"2024-04-24T22:39:38.708783+0000","last_peered":"2024-04-24T22:39:38.708783+0000","last_clean":"2024-04-24T22:39:38.708783+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:39:38.708783+0000","last_undegraded":"2024-04-24T22:39:38.708783+0000","last_fullsized":"2024-04-24T22:39:38.708783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706420+0000","last_change":"2024-04-24T22:39:37.700406+0000","last_active":"2024-04-24T22:39:38.706420+0000","last_peered":"2024-04-24T22:39:38.706420+0000","last_clean":"2024-04-24T22:39:38.706420+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:39:38.706420+0000","last_undegraded":"2024-04-24T22:39:38.706420+0000","last_fullsized":"2024-04-24T22:39:38.706420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819221+0000","last_change":"2024-04-24T22:39:36.701398+0000","last_active":"2024-04-24T22:39:38.819221+0000","last_peered":"2024-04-24T22:39:38.819221+0000","last_clean":"2024-04-24T22:39:38.819221+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:39:38.819221+0000","last_undegraded":"2024-04-24T22:39:38.819221+0000","last_fullsized":"2024-04-24T22:39:38.819221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708466+0000","last_change":"2024-04-24T22:39:36.712294+0000","last_active":"2024-04-24T22:39:38.708466+0000","last_peered":"2024-04-24T22:39:38.708466+0000","last_clean":"2024-04-24T22:39:38.708466+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:39:38.708466+0000","last_undegraded":"2024-04-24T22:39:38.708466+0000","last_fullsized":"2024-04-24T22:39:38.708466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707809+0000","last_change":"2024-04-24T22:39:37.716920+0000","last_active":"2024-04-24T22:39:38.707809+0000","last_peered":"2024-04-24T22:39:38.707809+0000","last_clean":"2024-04-24T22:39:38.707809+0000","last_became_active":"2024-04-24T22:39:37.716641+0000","last_became_peered":"2024-04-24T22:39:37.716641+0000","last_unstale":"2024-04-24T22:39:38.707809+0000","last_undegraded":"2024-04-24T22:39:38.707809+0000","last_fullsized":"2024-04-24T22:39:38.707809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820143+0000","last_change":"2024-04-24T22:39:36.698983+0000","last_active":"2024-04-24T22:39:38.820143+0000","last_peered":"2024-04-24T22:39:38.820143+0000","last_clean":"2024-04-24T22:39:38.820143+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:39:38.820143+0000","last_undegraded":"2024-04-24T22:39:38.820143+0000","last_fullsized":"2024-04-24T22:39:38.820143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820135+0000","last_change":"2024-04-24T22:39:37.714229+0000","last_active":"2024-04-24T22:39:38.820135+0000","last_peered":"2024-04-24T22:39:38.820135+0000","last_clean":"2024-04-24T22:39:38.820135+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:39:38.820135+0000","last_undegraded":"2024-04-24T22:39:38.820135+0000","last_fullsized":"2024-04-24T22:39:38.820135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819388+0000","last_change":"2024-04-24T22:39:36.705776+0000","last_active":"2024-04-24T22:39:38.819388+0000","last_peered":"2024-04-24T22:39:38.819388+0000","last_clean":"2024-04-24T22:39:38.819388+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:39:38.819388+0000","last_undegraded":"2024-04-24T22:39:38.819388+0000","last_fullsized":"2024-04-24T22:39:38.819388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820936+0000","last_change":"2024-04-24T22:39:37.701436+0000","last_active":"2024-04-24T22:39:38.820936+0000","last_peered":"2024-04-24T22:39:38.820936+0000","last_clean":"2024-04-24T22:39:38.820936+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:39:38.820936+0000","last_undegraded":"2024-04-24T22:39:38.820936+0000","last_fullsized":"2024-04-24T22:39:38.820936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820780+0000","last_change":"2024-04-24T22:39:36.700237+0000","last_active":"2024-04-24T22:39:38.820780+0000","last_peered":"2024-04-24T22:39:38.820780+0000","last_clean":"2024-04-24T22:39:38.820780+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:39:38.820780+0000","last_undegraded":"2024-04-24T22:39:38.820780+0000","last_fullsized":"2024-04-24T22:39:38.820780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705984+0000","last_change":"2024-04-24T22:39:37.703532+0000","last_active":"2024-04-24T22:39:38.705984+0000","last_peered":"2024-04-24T22:39:38.705984+0000","last_clean":"2024-04-24T22:39:38.705984+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:39:38.705984+0000","last_undegraded":"2024-04-24T22:39:38.705984+0000","last_fullsized":"2024-04-24T22:39:38.705984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820432+0000","last_change":"2024-04-24T22:39:36.717477+0000","last_active":"2024-04-24T22:39:38.820432+0000","last_peered":"2024-04-24T22:39:38.820432+0000","last_clean":"2024-04-24T22:39:38.820432+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:39:38.820432+0000","last_undegraded":"2024-04-24T22:39:38.820432+0000","last_fullsized":"2024-04-24T22:39:38.820432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819989+0000","last_change":"2024-04-24T22:39:37.699255+0000","last_active":"2024-04-24T22:39:38.819989+0000","last_peered":"2024-04-24T22:39:38.819989+0000","last_clean":"2024-04-24T22:39:38.819989+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:39:38.819989+0000","last_undegraded":"2024-04-24T22:39:38.819989+0000","last_fullsized":"2024-04-24T22:39:38.819989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819412+0000","last_change":"2024-04-24T22:39:36.706567+0000","last_active":"2024-04-24T22:39:38.819412+0000","last_peered":"2024-04-24T22:39:38.819412+0000","last_clean":"2024-04-24T22:39:38.819412+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:39:38.819412+0000","last_undegraded":"2024-04-24T22:39:38.819412+0000","last_fullsized":"2024-04-24T22:39:38.819412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706118+0000","last_change":"2024-04-24T22:39:37.712064+0000","last_active":"2024-04-24T22:39:38.706118+0000","last_peered":"2024-04-24T22:39:38.706118+0000","last_clean":"2024-04-24T22:39:38.706118+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:39:38.706118+0000","last_undegraded":"2024-04-24T22:39:38.706118+0000","last_fullsized":"2024-04-24T22:39:38.706118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706281+0000","last_change":"2024-04-24T22:39:36.700041+0000","last_active":"2024-04-24T22:39:38.706281+0000","last_peered":"2024-04-24T22:39:38.706281+0000","last_clean":"2024-04-24T22:39:38.706281+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:39:38.706281+0000","last_undegraded":"2024-04-24T22:39:38.706281+0000","last_fullsized":"2024-04-24T22:39:38.706281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:50.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705269+0000","last_change":"2024-04-24T22:39:37.691459+0000","last_active":"2024-04-24T22:39:38.705269+0000","last_peered":"2024-04-24T22:39:38.705269+0000","last_clean":"2024-04-24T22:39:38.705269+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:39:38.705269+0000","last_undegraded":"2024-04-24T22:39:38.705269+0000","last_fullsized":"2024-04-24T22:39:38.705269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820416+0000","last_change":"2024-04-24T22:39:36.706454+0000","last_active":"2024-04-24T22:39:38.820416+0000","last_peered":"2024-04-24T22:39:38.820416+0000","last_clean":"2024-04-24T22:39:38.820416+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:39:38.820416+0000","last_undegraded":"2024-04-24T22:39:38.820416+0000","last_fullsized":"2024-04-24T22:39:38.820416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705898+0000","last_change":"2024-04-24T22:39:37.696724+0000","last_active":"2024-04-24T22:39:38.705898+0000","last_peered":"2024-04-24T22:39:38.705898+0000","last_clean":"2024-04-24T22:39:38.705898+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:39:38.705898+0000","last_undegraded":"2024-04-24T22:39:38.705898+0000","last_fullsized":"2024-04-24T22:39:38.705898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818924+0000","last_change":"2024-04-24T22:39:36.701340+0000","last_active":"2024-04-24T22:39:38.818924+0000","last_peered":"2024-04-24T22:39:38.818924+0000","last_clean":"2024-04-24T22:39:38.818924+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:39:38.818924+0000","last_undegraded":"2024-04-24T22:39:38.818924+0000","last_fullsized":"2024-04-24T22:39:38.818924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819009+0000","last_change":"2024-04-24T22:39:37.697532+0000","last_active":"2024-04-24T22:39:38.819009+0000","last_peered":"2024-04-24T22:39:38.819009+0000","last_clean":"2024-04-24T22:39:38.819009+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:39:38.819009+0000","last_undegraded":"2024-04-24T22:39:38.819009+0000","last_fullsized":"2024-04-24T22:39:38.819009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819879+0000","last_change":"2024-04-24T22:39:36.724649+0000","last_active":"2024-04-24T22:39:38.819879+0000","last_peered":"2024-04-24T22:39:38.819879+0000","last_clean":"2024-04-24T22:39:38.819879+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:39:38.819879+0000","last_undegraded":"2024-04-24T22:39:38.819879+0000","last_fullsized":"2024-04-24T22:39:38.819879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705839+0000","last_change":"2024-04-24T22:39:37.702890+0000","last_active":"2024-04-24T22:39:38.705839+0000","last_peered":"2024-04-24T22:39:38.705839+0000","last_clean":"2024-04-24T22:39:38.705839+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:39:38.705839+0000","last_undegraded":"2024-04-24T22:39:38.705839+0000","last_fullsized":"2024-04-24T22:39:38.705839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819608+0000","last_change":"2024-04-24T22:39:36.716044+0000","last_active":"2024-04-24T22:39:38.819608+0000","last_peered":"2024-04-24T22:39:38.819608+0000","last_clean":"2024-04-24T22:39:38.819608+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:39:38.819608+0000","last_undegraded":"2024-04-24T22:39:38.819608+0000","last_fullsized":"2024-04-24T22:39:38.819608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:40.023933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706077+0000","last_change":"2024-04-24T22:39:37.702621+0000","last_active":"2024-04-24T22:39:38.706077+0000","last_peered":"2024-04-24T22:39:38.706077+0000","last_clean":"2024-04-24T22:39:38.706077+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:39:38.706077+0000","last_undegraded":"2024-04-24T22:39:38.706077+0000","last_fullsized":"2024-04-24T22:39:38.706077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819737+0000","last_change":"2024-04-24T22:39:36.714272+0000","last_active":"2024-04-24T22:39:38.819737+0000","last_peered":"2024-04-24T22:39:38.819737+0000","last_clean":"2024-04-24T22:39:38.819737+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:39:38.819737+0000","last_undegraded":"2024-04-24T22:39:38.819737+0000","last_fullsized":"2024-04-24T22:39:38.819737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706373+0000","last_change":"2024-04-24T22:39:37.703786+0000","last_active":"2024-04-24T22:39:38.706373+0000","last_peered":"2024-04-24T22:39:38.706373+0000","last_clean":"2024-04-24T22:39:38.706373+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:39:38.706373+0000","last_undegraded":"2024-04-24T22:39:38.706373+0000","last_fullsized":"2024-04-24T22:39:38.706373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:55.990656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820545+0000","last_change":"2024-04-24T22:39:36.715437+0000","last_active":"2024-04-24T22:39:38.820545+0000","last_peered":"2024-04-24T22:39:38.820545+0000","last_clean":"2024-04-24T22:39:38.820545+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:39:38.820545+0000","last_undegraded":"2024-04-24T22:39:38.820545+0000","last_fullsized":"2024-04-24T22:39:38.820545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820735+0000","last_change":"2024-04-24T22:39:37.715801+0000","last_active":"2024-04-24T22:39:38.820735+0000","last_peered":"2024-04-24T22:39:38.820735+0000","last_clean":"2024-04-24T22:39:38.820735+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:39:38.820735+0000","last_undegraded":"2024-04-24T22:39:38.820735+0000","last_fullsized":"2024-04-24T22:39:38.820735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819581+0000","last_change":"2024-04-24T22:39:36.710422+0000","last_active":"2024-04-24T22:39:38.819581+0000","last_peered":"2024-04-24T22:39:38.819581+0000","last_clean":"2024-04-24T22:39:38.819581+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:39:38.819581+0000","last_undegraded":"2024-04-24T22:39:38.819581+0000","last_fullsized":"2024-04-24T22:39:38.819581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706031+0000","last_change":"2024-04-24T22:39:37.706745+0000","last_active":"2024-04-24T22:39:38.706031+0000","last_peered":"2024-04-24T22:39:38.706031+0000","last_clean":"2024-04-24T22:39:38.706031+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:39:38.706031+0000","last_undegraded":"2024-04-24T22:39:38.706031+0000","last_fullsized":"2024-04-24T22:39:38.706031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705671+0000","last_change":"2024-04-24T22:39:32.657965+0000","last_active":"2024-04-24T22:39:38.705671+0000","last_peered":"2024-04-24T22:39:38.705671+0000","last_clean":"2024-04-24T22:39:38.705671+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:39:38.705671+0000","last_undegraded":"2024-04-24T22:39:38.705671+0000","last_fullsized":"2024-04-24T22:39:38.705671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_clean_scrub_stamp":"2024-04-24T22:39:31.635804+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.714153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820290+0000","last_change":"2024-04-24T22:39:36.710381+0000","last_active":"2024-04-24T22:39:38.820290+0000","last_peered":"2024-04-24T22:39:38.820290+0000","last_clean":"2024-04-24T22:39:38.820290+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:39:38.820290+0000","last_undegraded":"2024-04-24T22:39:38.820290+0000","last_fullsized":"2024-04-24T22:39:38.820290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708707+0000","last_change":"2024-04-24T22:39:37.713956+0000","last_active":"2024-04-24T22:39:38.708707+0000","last_peered":"2024-04-24T22:39:38.708707+0000","last_clean":"2024-04-24T22:39:38.708707+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:39:38.708707+0000","last_undegraded":"2024-04-24T22:39:38.708707+0000","last_fullsized":"2024-04-24T22:39:38.708707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705533+0000","last_change":"2024-04-24T22:39:36.715290+0000","last_active":"2024-04-24T22:39:38.705533+0000","last_peered":"2024-04-24T22:39:38.705533+0000","last_clean":"2024-04-24T22:39:38.705533+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:39:38.705533+0000","last_undegraded":"2024-04-24T22:39:38.705533+0000","last_fullsized":"2024-04-24T22:39:38.705533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704538+0000","last_change":"2024-04-24T22:39:37.695419+0000","last_active":"2024-04-24T22:39:38.704538+0000","last_peered":"2024-04-24T22:39:38.704538+0000","last_clean":"2024-04-24T22:39:38.704538+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:39:38.704538+0000","last_undegraded":"2024-04-24T22:39:38.704538+0000","last_fullsized":"2024-04-24T22:39:38.704538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:08.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819608+0000","last_change":"2024-04-24T22:39:36.707548+0000","last_active":"2024-04-24T22:39:38.819608+0000","last_peered":"2024-04-24T22:39:38.819608+0000","last_clean":"2024-04-24T22:39:38.819608+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:39:38.819608+0000","last_undegraded":"2024-04-24T22:39:38.819608+0000","last_fullsized":"2024-04-24T22:39:38.819608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820375+0000","last_change":"2024-04-24T22:39:37.703301+0000","last_active":"2024-04-24T22:39:38.820375+0000","last_peered":"2024-04-24T22:39:38.820375+0000","last_clean":"2024-04-24T22:39:38.820375+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:39:38.820375+0000","last_undegraded":"2024-04-24T22:39:38.820375+0000","last_fullsized":"2024-04-24T22:39:38.820375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820564+0000","last_change":"2024-04-24T22:39:36.713486+0000","last_active":"2024-04-24T22:39:38.820564+0000","last_peered":"2024-04-24T22:39:38.820564+0000","last_clean":"2024-04-24T22:39:38.820564+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:39:38.820564+0000","last_undegraded":"2024-04-24T22:39:38.820564+0000","last_fullsized":"2024-04-24T22:39:38.820564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819526+0000","last_change":"2024-04-24T22:39:37.705705+0000","last_active":"2024-04-24T22:39:38.819526+0000","last_peered":"2024-04-24T22:39:38.819526+0000","last_clean":"2024-04-24T22:39:38.819526+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:39:38.819526+0000","last_undegraded":"2024-04-24T22:39:38.819526+0000","last_fullsized":"2024-04-24T22:39:38.819526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:34.424877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820670+0000","last_change":"2024-04-24T22:39:36.723125+0000","last_active":"2024-04-24T22:39:38.820670+0000","last_peered":"2024-04-24T22:39:38.820670+0000","last_clean":"2024-04-24T22:39:38.820670+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:39:38.820670+0000","last_undegraded":"2024-04-24T22:39:38.820670+0000","last_fullsized":"2024-04-24T22:39:38.820670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820922+0000","last_change":"2024-04-24T22:39:37.713947+0000","last_active":"2024-04-24T22:39:38.820922+0000","last_peered":"2024-04-24T22:39:38.820922+0000","last_clean":"2024-04-24T22:39:38.820922+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:39:38.820922+0000","last_undegraded":"2024-04-24T22:39:38.820922+0000","last_fullsized":"2024-04-24T22:39:38.820922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706225+0000","last_change":"2024-04-24T22:39:36.716541+0000","last_active":"2024-04-24T22:39:38.706225+0000","last_peered":"2024-04-24T22:39:38.706225+0000","last_clean":"2024-04-24T22:39:38.706225+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:39:38.706225+0000","last_undegraded":"2024-04-24T22:39:38.706225+0000","last_fullsized":"2024-04-24T22:39:38.706225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:33.366773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820580+0000","last_change":"2024-04-24T22:39:37.708069+0000","last_active":"2024-04-24T22:39:38.820580+0000","last_peered":"2024-04-24T22:39:38.820580+0000","last_clean":"2024-04-24T22:39:38.820580+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:39:38.820580+0000","last_undegraded":"2024-04-24T22:39:38.820580+0000","last_fullsized":"2024-04-24T22:39:38.820580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.821341+0000","last_change":"2024-04-24T22:39:36.718740+0000","last_active":"2024-04-24T22:39:38.821341+0000","last_peered":"2024-04-24T22:39:38.821341+0000","last_clean":"2024-04-24T22:39:38.821341+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:39:38.821341+0000","last_undegraded":"2024-04-24T22:39:38.821341+0000","last_fullsized":"2024-04-24T22:39:38.821341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.709326+0000","last_change":"2024-04-24T22:39:37.704496+0000","last_active":"2024-04-24T22:39:38.709326+0000","last_peered":"2024-04-24T22:39:38.709326+0000","last_clean":"2024-04-24T22:39:38.709326+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:39:38.709326+0000","last_undegraded":"2024-04-24T22:39:38.709326+0000","last_fullsized":"2024-04-24T22:39:38.709326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819467+0000","last_change":"2024-04-24T22:39:36.701308+0000","last_active":"2024-04-24T22:39:38.819467+0000","last_peered":"2024-04-24T22:39:38.819467+0000","last_clean":"2024-04-24T22:39:38.819467+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:39:38.819467+0000","last_undegraded":"2024-04-24T22:39:38.819467+0000","last_fullsized":"2024-04-24T22:39:38.819467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705483+0000","last_change":"2024-04-24T22:39:37.700189+0000","last_active":"2024-04-24T22:39:38.705483+0000","last_peered":"2024-04-24T22:39:38.705483+0000","last_clean":"2024-04-24T22:39:38.705483+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:39:38.705483+0000","last_undegraded":"2024-04-24T22:39:38.705483+0000","last_fullsized":"2024-04-24T22:39:38.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705533+0000","last_change":"2024-04-24T22:39:36.716460+0000","last_active":"2024-04-24T22:39:38.705533+0000","last_peered":"2024-04-24T22:39:38.705533+0000","last_clean":"2024-04-24T22:39:38.705533+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:39:38.705533+0000","last_undegraded":"2024-04-24T22:39:38.705533+0000","last_fullsized":"2024-04-24T22:39:38.705533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819020+0000","last_change":"2024-04-24T22:39:37.704143+0000","last_active":"2024-04-24T22:39:38.819020+0000","last_peered":"2024-04-24T22:39:38.819020+0000","last_clean":"2024-04-24T22:39:38.819020+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:39:38.819020+0000","last_undegraded":"2024-04-24T22:39:38.819020+0000","last_fullsized":"2024-04-24T22:39:38.819020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:58.556519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820279+0000","last_change":"2024-04-24T22:39:36.721540+0000","last_active":"2024-04-24T22:39:38.820279+0000","last_peered":"2024-04-24T22:39:38.820279+0000","last_clean":"2024-04-24T22:39:38.820279+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:39:38.820279+0000","last_undegraded":"2024-04-24T22:39:38.820279+0000","last_fullsized":"2024-04-24T22:39:38.820279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708008+0000","last_change":"2024-04-24T22:39:37.714150+0000","last_active":"2024-04-24T22:39:38.708008+0000","last_peered":"2024-04-24T22:39:38.708008+0000","last_clean":"2024-04-24T22:39:38.708008+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:39:38.708008+0000","last_undegraded":"2024-04-24T22:39:38.708008+0000","last_fullsized":"2024-04-24T22:39:38.708008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820479+0000","last_change":"2024-04-24T22:39:36.716667+0000","last_active":"2024-04-24T22:39:38.820479+0000","last_peered":"2024-04-24T22:39:38.820479+0000","last_clean":"2024-04-24T22:39:38.820479+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:39:38.820479+0000","last_undegraded":"2024-04-24T22:39:38.820479+0000","last_fullsized":"2024-04-24T22:39:38.820479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708586+0000","last_change":"2024-04-24T22:39:37.715906+0000","last_active":"2024-04-24T22:39:38.708586+0000","last_peered":"2024-04-24T22:39:38.708586+0000","last_clean":"2024-04-24T22:39:38.708586+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:39:38.708586+0000","last_undegraded":"2024-04-24T22:39:38.708586+0000","last_fullsized":"2024-04-24T22:39:38.708586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706034+0000","last_change":"2024-04-24T22:39:36.698212+0000","last_active":"2024-04-24T22:39:38.706034+0000","last_peered":"2024-04-24T22:39:38.706034+0000","last_clean":"2024-04-24T22:39:38.706034+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:39:38.706034+0000","last_undegraded":"2024-04-24T22:39:38.706034+0000","last_fullsized":"2024-04-24T22:39:38.706034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818963+0000","last_change":"2024-04-24T22:39:37.696143+0000","last_active":"2024-04-24T22:39:38.818963+0000","last_peered":"2024-04-24T22:39:38.818963+0000","last_clean":"2024-04-24T22:39:38.818963+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:39:38.818963+0000","last_undegraded":"2024-04-24T22:39:38.818963+0000","last_fullsized":"2024-04-24T22:39:38.818963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820834+0000","last_change":"2024-04-24T22:39:37.696113+0000","last_active":"2024-04-24T22:39:38.820834+0000","last_peered":"2024-04-24T22:39:38.820834+0000","last_clean":"2024-04-24T22:39:38.820834+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:39:38.820834+0000","last_undegraded":"2024-04-24T22:39:38.820834+0000","last_fullsized":"2024-04-24T22:39:38.820834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705916+0000","last_change":"2024-04-24T22:39:36.699149+0000","last_active":"2024-04-24T22:39:38.705916+0000","last_peered":"2024-04-24T22:39:38.705916+0000","last_clean":"2024-04-24T22:39:38.705916+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:39:38.705916+0000","last_undegraded":"2024-04-24T22:39:38.705916+0000","last_fullsized":"2024-04-24T22:39:38.705916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820550+0000","last_change":"2024-04-24T22:39:37.700972+0000","last_active":"2024-04-24T22:39:38.820550+0000","last_peered":"2024-04-24T22:39:38.820550+0000","last_clean":"2024-04-24T22:39:38.820550+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:39:38.820550+0000","last_undegraded":"2024-04-24T22:39:38.820550+0000","last_fullsized":"2024-04-24T22:39:38.820550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706124+0000","last_change":"2024-04-24T22:39:36.698567+0000","last_active":"2024-04-24T22:39:38.706124+0000","last_peered":"2024-04-24T22:39:38.706124+0000","last_clean":"2024-04-24T22:39:38.706124+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:39:38.706124+0000","last_undegraded":"2024-04-24T22:39:38.706124+0000","last_fullsized":"2024-04-24T22:39:38.706124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706000+0000","last_change":"2024-04-24T22:39:37.706951+0000","last_active":"2024-04-24T22:39:38.706000+0000","last_peered":"2024-04-24T22:39:38.706000+0000","last_clean":"2024-04-24T22:39:38.706000+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:39:38.706000+0000","last_undegraded":"2024-04-24T22:39:38.706000+0000","last_fullsized":"2024-04-24T22:39:38.706000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708849+0000","last_change":"2024-04-24T22:39:36.703823+0000","last_active":"2024-04-24T22:39:38.708849+0000","last_peered":"2024-04-24T22:39:38.708849+0000","last_clean":"2024-04-24T22:39:38.708849+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:39:38.708849+0000","last_undegraded":"2024-04-24T22:39:38.708849+0000","last_fullsized":"2024-04-24T22:39:38.708849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708601+0000","last_change":"2024-04-24T22:39:37.712078+0000","last_active":"2024-04-24T22:39:38.708601+0000","last_peered":"2024-04-24T22:39:38.708601+0000","last_clean":"2024-04-24T22:39:38.708601+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:39:38.708601+0000","last_undegraded":"2024-04-24T22:39:38.708601+0000","last_fullsized":"2024-04-24T22:39:38.708601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706267+0000","last_change":"2024-04-24T22:39:36.719643+0000","last_active":"2024-04-24T22:39:38.706267+0000","last_peered":"2024-04-24T22:39:38.706267+0000","last_clean":"2024-04-24T22:39:38.706267+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:39:38.706267+0000","last_undegraded":"2024-04-24T22:39:38.706267+0000","last_fullsized":"2024-04-24T22:39:38.706267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":665973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":665973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:39:44.673 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T22:39:44.674 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T22:39:44.674 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-24T22:39:44.675 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-24T22:39:44.675 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-24T22:39:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.927+0000 7ffa2b600640 1 Processor -- start 2024-04-24T22:39:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.928+0000 7ffa2b600640 1 -- start start 2024-04-24T22:39:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.929+0000 7ffa2b600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa241545a0 0x7ffa2406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.930+0000 7ffa2b600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa2406b1d0 con 0x7ffa2406c700 2024-04-24T22:39:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.930+0000 7ffa2b600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa24073660 con 0x7ffa240730e0 2024-04-24T22:39:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.930+0000 7ffa2b600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ffa2415b600 con 0x7ffa241545a0 2024-04-24T22:39:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.930+0000 7ffa23e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffa2406c700 0x7ffa2415dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54448/0 (socket says 172.21.15.47:54448) 2024-04-24T22:39:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.930+0000 7ffa29600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ffa240730e0 0x7ffa24066ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:33106/0 (socket says 172.21.15.47:33106) 2024-04-24T22:39:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.930+0000 7ffa23e00640 1 -- 172.21.15.47:0/4264556707 learned_addr learned my addr 172.21.15.47:0/4264556707 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.930+0000 7ffa28c00640 1 --2- 172.21.15.47:0/4264556707 >> v2:172.21.15.113:3300/0 conn(0x7ffa241545a0 0x7ffa2406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.930+0000 7ffa23400640 1 -- 172.21.15.47:0/4264556707 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3040154775 0 0) 0x7ffa24073660 con 0x7ffa240730e0 2024-04-24T22:39:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.930+0000 7ffa23400640 1 -- 172.21.15.47:0/4264556707 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa0c003520 con 0x7ffa240730e0 2024-04-24T22:39:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.931+0000 7ffa23400640 1 -- 172.21.15.47:0/4264556707 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189590779 0 0) 0x7ffa2406b1d0 con 0x7ffa2406c700 2024-04-24T22:39:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.931+0000 7ffa23400640 1 -- 172.21.15.47:0/4264556707 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa24073660 con 0x7ffa2406c700 2024-04-24T22:39:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.931+0000 7ffa28c00640 1 -- 172.21.15.47:0/4264556707 >> v1:172.21.15.113:6789/0 conn(0x7ffa240730e0 legacy=0x7ffa24066ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.931+0000 7ffa28c00640 1 -- 172.21.15.47:0/4264556707 >> v1:172.21.15.47:6789/0 conn(0x7ffa2406c700 legacy=0x7ffa2415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.931+0000 7ffa28c00640 1 -- 172.21.15.47:0/4264556707 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2415a840 con 0x7ffa241545a0 2024-04-24T22:39:44.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.932+0000 7ffa28c00640 1 --2- 172.21.15.47:0/4264556707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 0x7ffa2406c1c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa180097a0 tx=0x7ffa1802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f214971303462ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:44.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.932+0000 7ffa23400640 1 -- 172.21.15.47:0/4264556707 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18037070 con 0x7ffa241545a0 2024-04-24T22:39:44.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.932+0000 7ffa23400640 1 -- 172.21.15.47:0/4264556707 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa18032440 con 0x7ffa241545a0 2024-04-24T22:39:44.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.932+0000 7ffa23400640 1 -- 172.21.15.47:0/4264556707 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18032810 con 0x7ffa241545a0 2024-04-24T22:39:44.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.933+0000 7ffa2b600640 1 -- 172.21.15.47:0/4264556707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 msgr2=0x7ffa2406c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.933+0000 7ffa2b600640 1 --2- 172.21.15.47:0/4264556707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 0x7ffa2406c1c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa180097a0 tx=0x7ffa1802a230 comp rx=0 tx=0).stop 2024-04-24T22:39:44.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.933+0000 7ffa2b600640 1 -- 172.21.15.47:0/4264556707 shutdown_connections 2024-04-24T22:39:44.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.933+0000 7ffa2b600640 1 --2- 172.21.15.47:0/4264556707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 0x7ffa2406c1c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.933+0000 7ffa2b600640 1 -- 172.21.15.47:0/4264556707 >> 172.21.15.47:0/4264556707 conn(0x7ffa240c9cb0 msgr2=0x7ffa240ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:44.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.933+0000 7ffa2b600640 1 -- 172.21.15.47:0/4264556707 shutdown_connections 2024-04-24T22:39:44.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.933+0000 7ffa2b600640 1 -- 172.21.15.47:0/4264556707 wait complete. 2024-04-24T22:39:44.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.935+0000 7ffa2b600640 1 Processor -- start 2024-04-24T22:39:44.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.935+0000 7ffa2b600640 1 -- start start 2024-04-24T22:39:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa2b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa2406c700 0x7ffa24157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa2b600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa240730e0 0x7ffa2406def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa2b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 0x7ffa2415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa2b600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2415e330 con 0x7ffa240730e0 2024-04-24T22:39:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa2b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2415ae00 con 0x7ffa241545a0 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa2b600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ffa241f0830 con 0x7ffa2406c700 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa29600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 0x7ffa2415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa240730e0 0x7ffa2406def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa29600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 0x7ffa2415be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:40730/0 (socket says 172.21.15.47:40730) 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa29600640 1 -- 172.21.15.47:0/2743721919 learned_addr learned my addr 172.21.15.47:0/2743721919 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.936+0000 7ffa28c00640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa2406c700 0x7ffa24157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa29600640 1 -- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa2406c700 msgr2=0x7ffa24157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa29600640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa2406c700 0x7ffa24157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa29600640 1 -- 172.21.15.47:0/2743721919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa240730e0 msgr2=0x7ffa2406def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa29600640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa240730e0 0x7ffa2406def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa29600640 1 -- 172.21.15.47:0/2743721919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa241efc40 con 0x7ffa241545a0 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa28c00640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa2406c700 0x7ffa24157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa23e00640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa240730e0 0x7ffa2406def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:44.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa29600640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 0x7ffa2415be70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00c700 tx=0x7ffa1c00cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:44.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa20c00640 1 -- 172.21.15.47:0/2743721919 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c017070 con 0x7ffa241545a0 2024-04-24T22:39:44.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa20c00640 1 -- 172.21.15.47:0/2743721919 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c00ce10 con 0x7ffa241545a0 2024-04-24T22:39:44.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa20c00640 1 -- 172.21.15.47:0/2743721919 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c00a880 con 0x7ffa241545a0 2024-04-24T22:39:44.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa2b600640 1 -- 172.21.15.47:0/2743721919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa241f3350 con 0x7ffa241545a0 2024-04-24T22:39:44.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.937+0000 7ffa2b600640 1 -- 172.21.15.47:0/2743721919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa241f01e0 con 0x7ffa241545a0 2024-04-24T22:39:44.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.938+0000 7ffa2b600640 1 -- 172.21.15.47:0/2743721919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa24073b90 con 0x7ffa241545a0 2024-04-24T22:39:44.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.939+0000 7ffa20c00640 1 -- 172.21.15.47:0/2743721919 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa1c01a070 con 0x7ffa241545a0 2024-04-24T22:39:44.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.940+0000 7ffa20c00640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ffa0406be30 0x7ffa0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:44.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.940+0000 7ffa20c00640 1 -- 172.21.15.47:0/2743721919 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffa1c08a4e0 con 0x7ffa241545a0 2024-04-24T22:39:44.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.940+0000 7ffa28c00640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ffa0406be30 0x7ffa0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:44.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.941+0000 7ffa28c00640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ffa0406be30 0x7ffa0406e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffa18034000 tx=0x7ffa1802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:44.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:44.945+0000 7ffa20c00640 1 -- 172.21.15.47:0/2743721919 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa1c058720 con 0x7ffa241545a0 2024-04-24T22:39:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.204+0000 7ffa2b600640 1 -- 172.21.15.47:0/2743721919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ffa2406bfa0 con 0x7ffa241545a0 2024-04-24T22:39:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.207+0000 7ffa20c00640 1 -- 172.21.15.47:0/2743721919 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172946 (secure 0 0 0) 0x7ffa1c05ca20 con 0x7ffa241545a0 2024-04-24T22:39:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 -- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ffa0406be30 msgr2=0x7ffa0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ffa0406be30 0x7ffa0406e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffa18034000 tx=0x7ffa1802e000 comp rx=0 tx=0).stop 2024-04-24T22:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 -- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 msgr2=0x7ffa2415be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 0x7ffa2415be70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00c700 tx=0x7ffa1c00cbc0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 -- 172.21.15.47:0/2743721919 shutdown_connections 2024-04-24T22:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ffa0406be30 0x7ffa0406e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa241545a0 0x7ffa2415be70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa240730e0 0x7ffa2406def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 --2- 172.21.15.47:0/2743721919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa2406c700 0x7ffa24157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 -- 172.21.15.47:0/2743721919 >> 172.21.15.47:0/2743721919 conn(0x7ffa240c9cb0 msgr2=0x7ffa240cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 -- 172.21.15.47:0/2743721919 shutdown_connections 2024-04-24T22:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.213+0000 7ffa2b600640 1 -- 172.21.15.47:0/2743721919 wait complete. 2024-04-24T22:39:45.234 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2101666090},{"type":"v1","addr":"172.21.15.113:6833","nonce":2101666090}]},"active_addr":"172.21.15.113:6833/2101666090","active_change":"2024-04-24T22:39:22.631120+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":4116966280}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1364706319}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1871077150}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":2226284400}]}]} 2024-04-24T22:39:45.237 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-24T22:39:45.237 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T22:39:45.238 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:45.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.445+0000 7f3d11600640 1 Processor -- start 2024-04-24T22:39:45.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.445+0000 7f3d11600640 1 -- start start 2024-04-24T22:39:45.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.447+0000 7f3d11600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 0x7f3d0c157420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.447+0000 7f3d11600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3d0c06b160 0x7f3d0c06b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.447+0000 7f3d11600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d0c0c28d0 con 0x7f3d0c0c5320 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.447+0000 7f3d11600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3d0c070ef0 con 0x7f3d0c06b160 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.447+0000 7f3d11600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0c0c26b0 con 0x7f3d0c0c2290 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.447+0000 7f3d0be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3d0c0c5320 0x7f3d0c0707e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54458/0 (socket says 172.21.15.47:54458) 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.447+0000 7f3d0be00640 1 -- 172.21.15.47:0/3774800294 learned_addr learned my addr 172.21.15.47:0/3774800294 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.447+0000 7f3d10c00640 1 --2- 172.21.15.47:0/3774800294 >> v2:172.21.15.47:3300/0 conn(0x7f3d0c06b160 0x7f3d0c06b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.448+0000 7f3d03e00640 1 --2- 172.21.15.47:0/3774800294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 0x7f3d0c157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.448+0000 7f3d0b400640 1 -- 172.21.15.47:0/3774800294 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2542088587 0 0) 0x7f3d0c0c28d0 con 0x7f3d0c0c5320 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.448+0000 7f3d0b400640 1 -- 172.21.15.47:0/3774800294 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cf0005730 con 0x7f3d0c0c5320 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.448+0000 7f3d03e00640 1 -- 172.21.15.47:0/3774800294 >> v2:172.21.15.47:3300/0 conn(0x7f3d0c06b160 msgr2=0x7f3d0c06b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.448+0000 7f3d03e00640 1 --2- 172.21.15.47:0/3774800294 >> v2:172.21.15.47:3300/0 conn(0x7f3d0c06b160 0x7f3d0c06b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.448+0000 7f3d03e00640 1 -- 172.21.15.47:0/3774800294 >> v1:172.21.15.47:6789/0 conn(0x7f3d0c0c5320 legacy=0x7f3d0c0707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.448+0000 7f3d03e00640 1 -- 172.21.15.47:0/3774800294 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c072080 con 0x7f3d0c0c2290 2024-04-24T22:39:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.448+0000 7f3d10c00640 1 --2- 172.21.15.47:0/3774800294 >> v2:172.21.15.47:3300/0 conn(0x7f3d0c06b160 0x7f3d0c06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.449+0000 7f3d03e00640 1 --2- 172.21.15.47:0/3774800294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 0x7f3d0c157420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40097a0 tx=0x7f3cf402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b18085db93a8c736 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.449+0000 7f3d0b400640 1 -- 172.21.15.47:0/3774800294 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4037070 con 0x7f3d0c0c2290 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.449+0000 7f3d0b400640 1 -- 172.21.15.47:0/3774800294 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf4032440 con 0x7f3d0c0c2290 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.449+0000 7f3d0b400640 1 -- 172.21.15.47:0/3774800294 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4032810 con 0x7f3d0c0c2290 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.450+0000 7f3d11600640 1 -- 172.21.15.47:0/3774800294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 msgr2=0x7f3d0c157420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.450+0000 7f3d11600640 1 --2- 172.21.15.47:0/3774800294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 0x7f3d0c157420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40097a0 tx=0x7f3cf402a230 comp rx=0 tx=0).stop 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.450+0000 7f3d11600640 1 -- 172.21.15.47:0/3774800294 shutdown_connections 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.450+0000 7f3d11600640 1 --2- 172.21.15.47:0/3774800294 >> v2:172.21.15.47:3300/0 conn(0x7f3d0c06b160 0x7f3d0c06b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.450+0000 7f3d11600640 1 --2- 172.21.15.47:0/3774800294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 0x7f3d0c157420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.450+0000 7f3d11600640 1 -- 172.21.15.47:0/3774800294 >> 172.21.15.47:0/3774800294 conn(0x7f3d0c0cb900 msgr2=0x7f3d0c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.450+0000 7f3d11600640 1 -- 172.21.15.47:0/3774800294 shutdown_connections 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.450+0000 7f3d11600640 1 -- 172.21.15.47:0/3774800294 wait complete. 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.452+0000 7f3d11600640 1 Processor -- start 2024-04-24T22:39:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.452+0000 7f3d11600640 1 -- start start 2024-04-24T22:39:45.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d11600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c06b160 0x7f3d0c14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d11600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 0x7f3d0c1df170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d11600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d0c0c5320 0x7f3d0c1df6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d11600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c151160 con 0x7f3d0c06b160 2024-04-24T22:39:45.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d11600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c157960 con 0x7f3d0c0c5320 2024-04-24T22:39:45.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d11600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0c072640 con 0x7f3d0c0c2290 2024-04-24T22:39:45.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c06b160 0x7f3d0c14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c06b160 0x7f3d0c14fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53694/0 (socket says 172.21.15.47:53694) 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d0be00640 1 -- 172.21.15.47:0/2272290265 learned_addr learned my addr 172.21.15.47:0/2272290265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d03e00640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 0x7f3d0c1df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.453+0000 7f3d10c00640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d0c0c5320 0x7f3d0c1df6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.454+0000 7f3d0be00640 1 -- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 msgr2=0x7f3d0c1df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.454+0000 7f3d0be00640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 0x7f3d0c1df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.454+0000 7f3d0be00640 1 -- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d0c0c5320 msgr2=0x7f3d0c1df6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.454+0000 7f3d0be00640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d0c0c5320 0x7f3d0c1df6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.454+0000 7f3d0be00640 1 -- 172.21.15.47:0/2272290265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c1e1660 con 0x7f3d0c06b160 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.454+0000 7f3d0be00640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c06b160 0x7f3d0c14fd50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00c3c0 tx=0x7f3cfc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.454+0000 7f3d08c00640 1 -- 172.21.15.47:0/2272290265 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc017070 con 0x7f3d0c06b160 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.454+0000 7f3d08c00640 1 -- 172.21.15.47:0/2272290265 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cfc005ce0 con 0x7f3d0c06b160 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.454+0000 7f3d08c00640 1 -- 172.21.15.47:0/2272290265 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0072d0 con 0x7f3d0c06b160 2024-04-24T22:39:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.454+0000 7f3d11600640 1 -- 172.21.15.47:0/2272290265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c0ca8e0 con 0x7f3d0c06b160 2024-04-24T22:39:45.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.455+0000 7f3d11600640 1 -- 172.21.15.47:0/2272290265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c148aa0 con 0x7f3d0c06b160 2024-04-24T22:39:45.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.456+0000 7f3d11600640 1 -- 172.21.15.47:0/2272290265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c068d50 con 0x7f3d0c06b160 2024-04-24T22:39:45.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.457+0000 7f3d08c00640 1 -- 172.21.15.47:0/2272290265 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cfc01a070 con 0x7f3d0c06b160 2024-04-24T22:39:45.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.458+0000 7f3d08c00640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ce806be60 0x7f3ce806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.458+0000 7f3d08c00640 1 -- 172.21.15.47:0/2272290265 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3cfc08a0c0 con 0x7f3d0c06b160 2024-04-24T22:39:45.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.458+0000 7f3d03e00640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ce806be60 0x7f3ce806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.459+0000 7f3d03e00640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ce806be60 0x7f3ce806e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4034000 tx=0x7f3cf402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:45.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.462+0000 7f3d08c00640 1 -- 172.21.15.47:0/2272290265 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3cfc058410 con 0x7f3d0c06b160 2024-04-24T22:39:45.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.672+0000 7f3d11600640 1 -- 172.21.15.47:0/2272290265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d0c04f5e0 con 0x7f3d0c06b160 2024-04-24T22:39:45.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.674+0000 7f3d08c00640 1 -- 172.21.15.47:0/2272290265 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16593 (secure 0 0 0) 0x7f3cfc05c710 con 0x7f3d0c06b160 2024-04-24T22:39:45.676 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:45.676 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:38.692613+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:45.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.677+0000 7f3d11600640 1 -- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ce806be60 msgr2=0x7f3ce806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.677+0000 7f3d11600640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ce806be60 0x7f3ce806e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4034000 tx=0x7f3cf402e000 comp rx=0 tx=0).stop 2024-04-24T22:39:45.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.678+0000 7f3d11600640 1 -- 172.21.15.47:0/2272290265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c06b160 msgr2=0x7f3d0c14fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.678+0000 7f3d11600640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c06b160 0x7f3d0c14fd50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00c3c0 tx=0x7f3cfc00c7c0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.678+0000 7f3d11600640 1 -- 172.21.15.47:0/2272290265 shutdown_connections 2024-04-24T22:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.678+0000 7f3d11600640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ce806be60 0x7f3ce806e320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.678+0000 7f3d11600640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d0c0c5320 0x7f3d0c1df6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.678+0000 7f3d11600640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d0c0c2290 0x7f3d0c1df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.678+0000 7f3d11600640 1 --2- 172.21.15.47:0/2272290265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3d0c06b160 0x7f3d0c14fd50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.678+0000 7f3d11600640 1 -- 172.21.15.47:0/2272290265 >> 172.21.15.47:0/2272290265 conn(0x7f3d0c0cb900 msgr2=0x7f3d0c0c6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.678+0000 7f3d11600640 1 -- 172.21.15.47:0/2272290265 shutdown_connections 2024-04-24T22:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.678+0000 7f3d11600640 1 -- 172.21.15.47:0/2272290265 wait complete. 2024-04-24T22:39:45.701 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T22:39:45.701 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:39:45.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.902+0000 7fc33c200640 1 Processor -- start 2024-04-24T22:39:45.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.903+0000 7fc33c200640 1 -- start start 2024-04-24T22:39:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.904+0000 7fc33c200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3340c7750 0x7fc3340c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.904+0000 7fc33c200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc334072b80 0x7fc334073010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.904+0000 7fc33c200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3340c7bb0 con 0x7fc334152960 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.904+0000 7fc33c200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc334158f20 con 0x7fc334072b80 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.904+0000 7fc33c200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc3341593b0 con 0x7fc3340c7750 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.905+0000 7fc33a200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc334072b80 0x7fc334073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.905+0000 7fc33a200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc334072b80 0x7fc334073010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:40748/0 (socket says 172.21.15.47:40748) 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.905+0000 7fc33a200640 1 -- 172.21.15.47:0/1607825863 learned_addr learned my addr 172.21.15.47:0/1607825863 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.905+0000 7fc338e00640 1 --2- 172.21.15.47:0/1607825863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3340c7750 0x7fc3340c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.905+0000 7fc33a200640 1 -- 172.21.15.47:0/1607825863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3340c7750 msgr2=0x7fc3340c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.905+0000 7fc33a200640 1 --2- 172.21.15.47:0/1607825863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3340c7750 0x7fc3340c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.905+0000 7fc33a200640 1 -- 172.21.15.47:0/1607825863 >> v1:172.21.15.47:6789/0 conn(0x7fc334152960 legacy=0x7fc33415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.905+0000 7fc33a200640 1 -- 172.21.15.47:0/1607825863 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3340c7e50 con 0x7fc334072b80 2024-04-24T22:39:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.905+0000 7fc338e00640 1 --2- 172.21.15.47:0/1607825863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3340c7750 0x7fc3340c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.906+0000 7fc33a200640 1 --2- 172.21.15.47:0/1607825863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc334072b80 0x7fc334073010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009e50 tx=0x7fc32c02a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7908af31ee2afed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.907+0000 7fc333e00640 1 -- 172.21.15.47:0/1607825863 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c036070 con 0x7fc334072b80 2024-04-24T22:39:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.907+0000 7fc333e00640 1 -- 172.21.15.47:0/1607825863 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc32c031600 con 0x7fc334072b80 2024-04-24T22:39:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.907+0000 7fc333e00640 1 -- 172.21.15.47:0/1607825863 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0319d0 con 0x7fc334072b80 2024-04-24T22:39:45.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.908+0000 7fc33c200640 1 -- 172.21.15.47:0/1607825863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc334072b80 msgr2=0x7fc334073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.908+0000 7fc33c200640 1 --2- 172.21.15.47:0/1607825863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc334072b80 0x7fc334073010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009e50 tx=0x7fc32c02a7a0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.908+0000 7fc33c200640 1 -- 172.21.15.47:0/1607825863 shutdown_connections 2024-04-24T22:39:45.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.908+0000 7fc33c200640 1 --2- 172.21.15.47:0/1607825863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc334072b80 0x7fc334073010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.908+0000 7fc33c200640 1 --2- 172.21.15.47:0/1607825863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3340c7750 0x7fc3340c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.908+0000 7fc33c200640 1 -- 172.21.15.47:0/1607825863 >> 172.21.15.47:0/1607825863 conn(0x7fc33406e580 msgr2=0x7fc33406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:45.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.908+0000 7fc33c200640 1 -- 172.21.15.47:0/1607825863 shutdown_connections 2024-04-24T22:39:45.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.908+0000 7fc33c200640 1 -- 172.21.15.47:0/1607825863 wait complete. 2024-04-24T22:39:45.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.910+0000 7fc33c200640 1 Processor -- start 2024-04-24T22:39:45.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.910+0000 7fc33c200640 1 -- start start 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc33c200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc334072b80 0x7fc3341f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc33c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3340c7750 0x7fc334159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc33c200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334152960 0x7fc3340c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc33c200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341588d0 con 0x7fc334152960 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc33c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc3340c8130 con 0x7fc3340c7750 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc33c200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc334158bb0 con 0x7fc334072b80 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc338e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3340c7750 0x7fc334159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc33a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334152960 0x7fc3340c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc338e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3340c7750 0x7fc334159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:40758/0 (socket says 172.21.15.47:40758) 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc338e00640 1 -- 172.21.15.47:0/3724620792 learned_addr learned my addr 172.21.15.47:0/3724620792 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc339800640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc334072b80 0x7fc3341f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc338e00640 1 -- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc334072b80 msgr2=0x7fc3341f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc338e00640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc334072b80 0x7fc3341f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc338e00640 1 -- 172.21.15.47:0/3724620792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334152960 msgr2=0x7fc3340c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc338e00640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334152960 0x7fc3340c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.912+0000 7fc338e00640 1 -- 172.21.15.47:0/3724620792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33415bf20 con 0x7fc3340c7750 2024-04-24T22:39:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.913+0000 7fc338e00640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3340c7750 0x7fc334159860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc32000e7e0 tx=0x7fc32000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.913+0000 7fc331600640 1 -- 172.21.15.47:0/3724620792 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32000f040 con 0x7fc3340c7750 2024-04-24T22:39:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.913+0000 7fc331600640 1 -- 172.21.15.47:0/3724620792 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc320004030 con 0x7fc3340c7750 2024-04-24T22:39:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.913+0000 7fc331600640 1 -- 172.21.15.47:0/3724620792 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320004400 con 0x7fc3340c7750 2024-04-24T22:39:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.913+0000 7fc33c200640 1 -- 172.21.15.47:0/3724620792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3341f3b00 con 0x7fc3340c7750 2024-04-24T22:39:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.913+0000 7fc33c200640 1 -- 172.21.15.47:0/3724620792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc33415b960 con 0x7fc3340c7750 2024-04-24T22:39:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.914+0000 7fc33c200640 1 -- 172.21.15.47:0/3724620792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc334073b90 con 0x7fc3340c7750 2024-04-24T22:39:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.915+0000 7fc331600640 1 -- 172.21.15.47:0/3724620792 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc32000c6d0 con 0x7fc3340c7750 2024-04-24T22:39:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.916+0000 7fc331600640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc31406be30 0x7fc31406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.916+0000 7fc331600640 1 -- 172.21.15.47:0/3724620792 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc32008ab60 con 0x7fc3340c7750 2024-04-24T22:39:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.916+0000 7fc339800640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc31406be30 0x7fc31406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.917+0000 7fc339800640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc31406be30 0x7fc31406e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3280016e0 tx=0x7fc328009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:45.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:45.920+0000 7fc331600640 1 -- 172.21.15.47:0/3724620792 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc320017020 con 0x7fc3340c7750 2024-04-24T22:39:46.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.123+0000 7fc33c200640 1 -- 172.21.15.47:0/3724620792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc334151d40 con 0x7fc3340c7750 2024-04-24T22:39:46.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.125+0000 7fc331600640 1 -- 172.21.15.47:0/3724620792 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16593 (secure 0 0 0) 0x7fc320058ef0 con 0x7fc3340c7750 2024-04-24T22:39:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:38.692613+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:39:46.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 -- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc31406be30 msgr2=0x7fc31406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc31406be30 0x7fc31406e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3280016e0 tx=0x7fc328009040 comp rx=0 tx=0).stop 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 -- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3340c7750 msgr2=0x7fc334159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3340c7750 0x7fc334159860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc32000e7e0 tx=0x7fc32000eca0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 -- 172.21.15.47:0/3724620792 shutdown_connections 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc31406be30 0x7fc31406e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334152960 0x7fc3340c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3340c7750 0x7fc334159860 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 --2- 172.21.15.47:0/3724620792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc334072b80 0x7fc3341f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 -- 172.21.15.47:0/3724620792 >> 172.21.15.47:0/3724620792 conn(0x7fc33406e580 msgr2=0x7fc334150030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 -- 172.21.15.47:0/3724620792 shutdown_connections 2024-04-24T22:39:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.128+0000 7fc33c200640 1 -- 172.21.15.47:0/3724620792 wait complete. 2024-04-24T22:39:46.150 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T22:39:46.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T22:39:46.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T22:39:46.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T22:39:46.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T22:39:46.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T22:39:46.153 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T22:39:46.153 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T22:39:46.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.444+0000 7f623a800640 1 Processor -- start 2024-04-24T22:39:46.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.444+0000 7f623a800640 1 -- start start 2024-04-24T22:39:46.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.447+0000 7f8b5cc00640 1 Processor -- start 2024-04-24T22:39:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.446+0000 7f623a800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f622c037670 0x7f622c04f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.446+0000 7f623a800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f622c037a50 0x7f622c0e3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.446+0000 7f623a800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f622c007a90 con 0x7f622c007670 2024-04-24T22:39:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.446+0000 7f623a800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f622c0071e0 con 0x7f622c037a50 2024-04-24T22:39:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.447+0000 7f623a800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f622c039490 con 0x7f622c037670 2024-04-24T22:39:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.447+0000 7f8b5cc00640 1 -- start start 2024-04-24T22:39:46.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.448+0000 7f6238a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f622c007670 0x7f622c038d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40858/0 (socket says 172.21.15.47:40858) 2024-04-24T22:39:46.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.448+0000 7f6239400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f622c037670 0x7f622c04f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.448+0000 7f6239e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f622c037a50 0x7f622c0e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.448+0000 7f6239e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f622c037a50 0x7f622c0e3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53882/0 (socket says 172.21.15.47:53882) 2024-04-24T22:39:46.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.448+0000 7f6239e00640 1 -- 172.21.15.47:0/227576604 learned_addr learned my addr 172.21.15.47:0/227576604 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.449+0000 7f6239e00640 1 -- 172.21.15.47:0/227576604 >> v2:172.21.15.113:3300/0 conn(0x7f622c037670 msgr2=0x7f622c04f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.449+0000 7f8b5cc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b50006f30 0x7f8b500e33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.449+0000 7f8b5cc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b500e6920 0x7f8b50007310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.449+0000 7f8b5cc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b5004b9b0 con 0x7f8b50007850 2024-04-24T22:39:46.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.449+0000 7f8b5cc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8b5004bce0 con 0x7f8b50006f30 2024-04-24T22:39:46.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.449+0000 7f8b5cc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8b500e39f0 con 0x7f8b500e6920 2024-04-24T22:39:46.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.449+0000 7f6239e00640 1 --2- 172.21.15.47:0/227576604 >> v2:172.21.15.113:3300/0 conn(0x7f622c037670 0x7f622c04f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.449+0000 7f6239e00640 1 -- 172.21.15.47:0/227576604 >> v1:172.21.15.47:6789/0 conn(0x7f622c007670 legacy=0x7f622c038d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.450+0000 7f8b57e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8b50007850 0x7f8b5004c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:51720/0 (socket says 172.21.15.47:51720) 2024-04-24T22:39:46.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.450+0000 7f8b57e00640 1 -- 172.21.15.47:0/1354817924 learned_addr learned my addr 172.21.15.47:0/1354817924 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:46.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.450+0000 7f8b57400640 1 --2- 172.21.15.47:0/1354817924 >> v2:172.21.15.113:3300/0 conn(0x7f8b50006f30 0x7f8b500e33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.449+0000 7f6239e00640 1 -- 172.21.15.47:0/227576604 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622c0062f0 con 0x7f622c037a50 2024-04-24T22:39:46.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.450+0000 7f8b56a00640 1 --2- 172.21.15.47:0/1354817924 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b500e6920 0x7f8b50007310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.450+0000 7f8b56000640 1 -- 172.21.15.47:0/1354817924 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4084152901 0 0) 0x7f8b5004b9b0 con 0x7f8b50007850 2024-04-24T22:39:46.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.450+0000 7f8b56000640 1 -- 172.21.15.47:0/1354817924 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b40003520 con 0x7f8b50007850 2024-04-24T22:39:46.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.450+0000 7f8b56a00640 1 -- 172.21.15.47:0/1354817924 >> v2:172.21.15.113:3300/0 conn(0x7f8b50006f30 msgr2=0x7f8b500e33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.450+0000 7f8b56a00640 1 --2- 172.21.15.47:0/1354817924 >> v2:172.21.15.113:3300/0 conn(0x7f8b50006f30 0x7f8b500e33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.450+0000 7f8b56a00640 1 -- 172.21.15.47:0/1354817924 >> v1:172.21.15.113:6789/0 conn(0x7f8b50007850 legacy=0x7f8b5004c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.451+0000 7f8b56a00640 1 -- 172.21.15.47:0/1354817924 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b500e3cd0 con 0x7f8b500e6920 2024-04-24T22:39:46.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.451+0000 7f6239e00640 1 --2- 172.21.15.47:0/227576604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c037a50 0x7f622c0e3880 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f62340502b0 tx=0x7f62340a6560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4a1347df4ade4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.452+0000 7f6233e00640 1 -- 172.21.15.47:0/227576604 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623406d040 con 0x7f622c037a50 2024-04-24T22:39:46.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.452+0000 7f6233e00640 1 -- 172.21.15.47:0/227576604 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62340ad640 con 0x7f622c037a50 2024-04-24T22:39:46.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.453+0000 7f8b56a00640 1 --2- 172.21.15.47:0/1354817924 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b500e6920 0x7f8b50007310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8b440097d0 tx=0x7f8b4402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d227ce9b1e3e71dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.452+0000 7f6233e00640 1 -- 172.21.15.47:0/227576604 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62340adab0 con 0x7f622c037a50 2024-04-24T22:39:46.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.453+0000 7f623a800640 1 -- 172.21.15.47:0/227576604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c037a50 msgr2=0x7f622c0e3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.453+0000 7f623a800640 1 --2- 172.21.15.47:0/227576604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c037a50 0x7f622c0e3880 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f62340502b0 tx=0x7f62340a6560 comp rx=0 tx=0).stop 2024-04-24T22:39:46.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7fe28a000640 1 Processor -- start 2024-04-24T22:39:46.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7f8b56000640 1 -- 172.21.15.47:0/1354817924 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44037070 con 0x7f8b500e6920 2024-04-24T22:39:46.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7f8b56000640 1 -- 172.21.15.47:0/1354817924 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b44032440 con 0x7f8b500e6920 2024-04-24T22:39:46.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7f8b56000640 1 -- 172.21.15.47:0/1354817924 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44032850 con 0x7f8b500e6920 2024-04-24T22:39:46.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7f623a800640 1 -- 172.21.15.47:0/227576604 shutdown_connections 2024-04-24T22:39:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7f623a800640 1 --2- 172.21.15.47:0/227576604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c037a50 0x7f622c0e3880 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7f623a800640 1 --2- 172.21.15.47:0/227576604 >> v2:172.21.15.113:3300/0 conn(0x7f622c037670 0x7f622c04f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7f623a800640 1 -- 172.21.15.47:0/227576604 >> 172.21.15.47:0/227576604 conn(0x7f622c054610 msgr2=0x7f622c054a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7fe28a000640 1 -- start start 2024-04-24T22:39:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.455+0000 7fe28a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe27c04f820 0x7fe27c0df7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.456+0000 7fe28a000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe27c007540 0x7fe27c04fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.456+0000 7fe28a000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe27c0060e0 con 0x7fe27c0d8ff0 2024-04-24T22:39:46.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.456+0000 7fe28a000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe27c005e90 con 0x7fe27c007540 2024-04-24T22:39:46.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.456+0000 7fe28a000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe27c006cf0 con 0x7fe27c04f820 2024-04-24T22:39:46.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.456+0000 7f623a800640 1 -- 172.21.15.47:0/227576604 shutdown_connections 2024-04-24T22:39:46.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.456+0000 7fe283e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe27c0d8ff0 0x7fe27c050970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:51726/0 (socket says 172.21.15.47:51726) 2024-04-24T22:39:46.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.456+0000 7fe283e00640 1 -- 172.21.15.47:0/357223353 learned_addr learned my addr 172.21.15.47:0/357223353 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:46.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.457+0000 7fe288c00640 1 --2- 172.21.15.47:0/357223353 >> v2:172.21.15.113:3300/0 conn(0x7fe27c04f820 0x7fe27c0df7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.457+0000 7fe283400640 1 -- 172.21.15.47:0/357223353 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3992030070 0 0) 0x7fe27c0060e0 con 0x7fe27c0d8ff0 2024-04-24T22:39:46.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.456+0000 7f623a800640 1 -- 172.21.15.47:0/227576604 wait complete. 2024-04-24T22:39:46.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.457+0000 7fe283400640 1 -- 172.21.15.47:0/357223353 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe26c003520 con 0x7fe27c0d8ff0 2024-04-24T22:39:46.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.457+0000 7fe289600640 1 --2- 172.21.15.47:0/357223353 >> v2:172.21.15.47:3300/0 conn(0x7fe27c007540 0x7fe27c04fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1354817924 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b500e6920 msgr2=0x7f8b50007310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.454+0000 7f8b5cc00640 1 --2- 172.21.15.47:0/1354817924 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b500e6920 0x7f8b50007310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8b440097d0 tx=0x7f8b4402a290 comp rx=0 tx=0).stop 2024-04-24T22:39:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.458+0000 7fe288c00640 1 -- 172.21.15.47:0/357223353 >> v2:172.21.15.47:3300/0 conn(0x7fe27c007540 msgr2=0x7fe27c04fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.458+0000 7fe283400640 1 -- 172.21.15.47:0/357223353 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 972571008 0 0) 0x7fe26c003520 con 0x7fe27c0d8ff0 2024-04-24T22:39:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.455+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1354817924 shutdown_connections 2024-04-24T22:39:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.455+0000 7f8b5cc00640 1 --2- 172.21.15.47:0/1354817924 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b500e6920 0x7f8b50007310 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.455+0000 7f8b5cc00640 1 --2- 172.21.15.47:0/1354817924 >> v2:172.21.15.113:3300/0 conn(0x7f8b50006f30 0x7f8b500e33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.455+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1354817924 >> 172.21.15.47:0/1354817924 conn(0x7f8b500061a0 msgr2=0x7f8b500065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.456+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1354817924 shutdown_connections 2024-04-24T22:39:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.456+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1354817924 wait complete. 2024-04-24T22:39:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.458+0000 7fe288c00640 1 --2- 172.21.15.47:0/357223353 >> v2:172.21.15.47:3300/0 conn(0x7fe27c007540 0x7fe27c04fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.458+0000 7fe288c00640 1 -- 172.21.15.47:0/357223353 >> v1:172.21.15.113:6789/0 conn(0x7fe27c0d8ff0 legacy=0x7fe27c050970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.458+0000 7fe288c00640 1 -- 172.21.15.47:0/357223353 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c0063c0 con 0x7fe27c04f820 2024-04-24T22:39:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.458+0000 7fe289600640 1 --2- 172.21.15.47:0/357223353 >> v2:172.21.15.47:3300/0 conn(0x7fe27c007540 0x7fe27c04fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:46.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.458+0000 7f8b5cc00640 1 Processor -- start 2024-04-24T22:39:46.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.458+0000 7f8b5cc00640 1 -- start start 2024-04-24T22:39:46.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.459+0000 7f623a800640 1 Processor -- start 2024-04-24T22:39:46.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.460+0000 7f623a800640 1 -- start start 2024-04-24T22:39:46.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.460+0000 7fe288c00640 1 --2- 172.21.15.47:0/357223353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c04f820 0x7fe27c0df7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe27c051f00 tx=0x7fe27802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df72dd4dde76583d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.460+0000 7f8b5cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b50006f30 0x7f8b500df740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.460+0000 7f8b5cc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b500e6920 0x7f8b500ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.460+0000 7f8b5cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b5004ee70 0x7f8b50181df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.460+0000 7f8b5cc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5004c880 con 0x7f8b50006f30 2024-04-24T22:39:46.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.460+0000 7f8b5cc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b500e4290 con 0x7f8b5004ee70 2024-04-24T22:39:46.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.460+0000 7f8b5cc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5004ca00 con 0x7f8b500e6920 2024-04-24T22:39:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.460+0000 7fe283400640 1 -- 172.21.15.47:0/357223353 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278037070 con 0x7fe27c04f820 2024-04-24T22:39:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7fe283400640 1 -- 172.21.15.47:0/357223353 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe27802abc0 con 0x7fe27c04f820 2024-04-24T22:39:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f8b57400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b50006f30 0x7f8b500df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7fe283400640 1 -- 172.21.15.47:0/357223353 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270001670 con 0x7fe27c04f820 2024-04-24T22:39:46.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f0866a00640 1 Processor -- start 2024-04-24T22:39:46.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f8b57400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b50006f30 0x7f8b500df740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53906/0 (socket says 172.21.15.47:53906) 2024-04-24T22:39:46.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f8b57400640 1 -- 172.21.15.47:0/1386299925 learned_addr learned my addr 172.21.15.47:0/1386299925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f623a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c007670 0x7f622c0dab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f0866a00640 1 -- start start 2024-04-24T22:39:46.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f623a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f622c037670 0x7f622c0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f623a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c037a50 0x7f622c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f623a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f622c006b90 con 0x7f622c037a50 2024-04-24T22:39:46.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7f8b57400640 1 -- 172.21.15.47:0/1386299925 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b500e6920 msgr2=0x7f8b500ddd00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:39:46.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7f8b57400640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b500e6920 0x7f8b500ddd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7f8b57400640 1 -- 172.21.15.47:0/1386299925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b5004ee70 msgr2=0x7f8b50181df0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:39:46.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f623a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f622c0068b0 con 0x7f622c007670 2024-04-24T22:39:46.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7f8b57400640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b5004ee70 0x7f8b50181df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7f8b57400640 1 -- 172.21.15.47:0/1386299925 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b500e0d30 con 0x7f8b50006f30 2024-04-24T22:39:46.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7fe28a000640 1 -- 172.21.15.47:0/357223353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c04f820 msgr2=0x7fe27c0df7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.461+0000 7f623a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f622c006e70 con 0x7f622c037670 2024-04-24T22:39:46.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7f6238a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f622c037670 0x7f622c0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7f6238a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f622c037670 0x7f622c0ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:59014/0 (socket says 172.21.15.47:59014) 2024-04-24T22:39:46.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7f6238a00640 1 -- 172.21.15.47:0/3031054797 learned_addr learned my addr 172.21.15.47:0/3031054797 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7fe28a000640 1 --2- 172.21.15.47:0/357223353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c04f820 0x7fe27c0df7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe27c051f00 tx=0x7fe27802a130 comp rx=0 tx=0).stop 2024-04-24T22:39:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.462+0000 7f8b57400640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b50006f30 0x7f8b500df740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00ec40 tx=0x7f8b4c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7fe28a000640 1 -- 172.21.15.47:0/357223353 shutdown_connections 2024-04-24T22:39:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7fe28a000640 1 --2- 172.21.15.47:0/357223353 >> v2:172.21.15.47:3300/0 conn(0x7fe27c007540 0x7fe27c04fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7fe28a000640 1 --2- 172.21.15.47:0/357223353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c04f820 0x7fe27c0df7a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7fe28a000640 1 -- 172.21.15.47:0/357223353 >> 172.21.15.47:0/357223353 conn(0x7fe27c00ee70 msgr2=0x7fe27c010290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7fe28a000640 1 -- 172.21.15.47:0/357223353 shutdown_connections 2024-04-24T22:39:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7f6239e00640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c037a50 0x7f622c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7f6239400640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c007670 0x7f622c0dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7f8b4b400640 1 -- 172.21.15.47:0/1386299925 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c017070 con 0x7f8b50006f30 2024-04-24T22:39:46.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7f8b4b400640 1 -- 172.21.15.47:0/1386299925 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4c00ed30 con 0x7f8b50006f30 2024-04-24T22:39:46.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7f8b4b400640 1 -- 172.21.15.47:0/1386299925 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00ce10 con 0x7f8b50006f30 2024-04-24T22:39:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7f6238a00640 1 -- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c007670 msgr2=0x7f622c0dab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7f6238a00640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f622c007670 0x7f622c0dab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7f6238a00640 1 -- 172.21.15.47:0/3031054797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c037a50 msgr2=0x7f622c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7f6238a00640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f622c037a50 0x7f622c0cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7f6238a00640 1 -- 172.21.15.47:0/3031054797 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622c048050 con 0x7f622c037670 2024-04-24T22:39:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.463+0000 7fe28a000640 1 -- 172.21.15.47:0/357223353 wait complete. 2024-04-24T22:39:46.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.464+0000 7f0866a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0860150ab0 0x7f08600c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.464+0000 7f6238a00640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f622c037670 0x7f622c0ca880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f622000c3e0 tx=0x7f622000c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.464+0000 7f0866a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086015a960 0x7f0860156770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.464+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b50181330 con 0x7f8b50006f30 2024-04-24T22:39:46.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.464+0000 7f6231600640 1 -- 172.21.15.47:0/3031054797 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200074e0 con 0x7f622c037670 2024-04-24T22:39:46.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.465+0000 7f6231600640 1 -- 172.21.15.47:0/3031054797 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6220007680 con 0x7f622c037670 2024-04-24T22:39:46.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.465+0000 7f6231600640 1 -- 172.21.15.47:0/3031054797 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200074e0 con 0x7f622c037670 2024-04-24T22:39:46.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.464+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5004f850 con 0x7f8b50006f30 2024-04-24T22:39:46.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.464+0000 7f085fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0860150ab0 0x7f08600c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.465+0000 7f085fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0860150ab0 0x7f08600c0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37726/0 (socket says 172.21.15.47:37726) 2024-04-24T22:39:46.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.465+0000 7f085fe00640 1 -- 172.21.15.47:0/3263870323 learned_addr learned my addr 172.21.15.47:0/3263870323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.465+0000 7f085f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086015a960 0x7f0860156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.465+0000 7fe28a000640 1 Processor -- start 2024-04-24T22:39:46.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.466+0000 7f0866a00640 1 -- 172.21.15.47:0/3263870323 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08600c14f0 con 0x7f08600c7940 2024-04-24T22:39:46.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.466+0000 7fe28a000640 1 -- start start 2024-04-24T22:39:46.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.466+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622c0c9f90 con 0x7f622c037670 2024-04-24T22:39:46.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.466+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f622c0ca550 con 0x7f622c037670 2024-04-24T22:39:46.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.466+0000 7f0866a00640 1 -- 172.21.15.47:0/3263870323 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f08600c17d0 con 0x7f0860150ab0 2024-04-24T22:39:46.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.466+0000 7f0866a00640 1 -- 172.21.15.47:0/3263870323 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0860159e80 con 0x7f086015a960 2024-04-24T22:39:46.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.467+0000 7f085f400640 1 -- 172.21.15.47:0/3263870323 >> v2:172.21.15.113:3300/0 conn(0x7f0860150ab0 msgr2=0x7f08600c0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.468+0000 7f8b4b400640 1 -- 172.21.15.47:0/1386299925 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b4c01a070 con 0x7f8b50006f30 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.467+0000 7fe28a000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe27c007540 0x7fe27c0d57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.467+0000 7fe28a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe27c04f820 0x7fe27c0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.467+0000 7fe28a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0dd600 0x7fe27c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.468+0000 7fe28a000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c051080 con 0x7fe27c04f820 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.468+0000 7fe28a000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c006980 con 0x7fe27c0dd600 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.468+0000 7fe28a000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe27c051200 con 0x7fe27c007540 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.468+0000 7fe283e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe27c04f820 0x7fe27c0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.468+0000 7fe289600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0dd600 0x7fe27c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.468+0000 7fe289600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0dd600 0x7fe27c0ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37736/0 (socket says 172.21.15.47:37736) 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.468+0000 7fe289600640 1 -- 172.21.15.47:0/1967708990 learned_addr learned my addr 172.21.15.47:0/1967708990 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.468+0000 7fe288c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe27c007540 0x7fe27c0d57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.467+0000 7f085f400640 1 --2- 172.21.15.47:0/3263870323 >> v2:172.21.15.113:3300/0 conn(0x7f0860150ab0 0x7f08600c0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.467+0000 7f085f400640 1 -- 172.21.15.47:0/3263870323 >> v1:172.21.15.113:6789/0 conn(0x7f08600c7940 legacy=0x7f08600c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.469+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b500057e0 con 0x7f8b50006f30 2024-04-24T22:39:46.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.467+0000 7f085f400640 1 -- 172.21.15.47:0/3263870323 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086015a160 con 0x7f086015a960 2024-04-24T22:39:46.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.469+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f622c005390 con 0x7f622c037670 2024-04-24T22:39:46.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.467+0000 7f085fe00640 1 --2- 172.21.15.47:0/3263870323 >> v2:172.21.15.113:3300/0 conn(0x7f0860150ab0 0x7f08600c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:46.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.467+0000 7f085ea00640 1 -- 172.21.15.47:0/3263870323 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2685584520 0 0) 0x7f08600c14f0 con 0x7f08600c7940 2024-04-24T22:39:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.469+0000 7fe289600640 1 -- 172.21.15.47:0/1967708990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe27c007540 msgr2=0x7fe27c0d57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.469+0000 7fe289600640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe27c007540 0x7fe27c0d57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.469+0000 7fe289600640 1 -- 172.21.15.47:0/1967708990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe27c04f820 msgr2=0x7fe27c0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.469+0000 7fe289600640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe27c04f820 0x7fe27c0ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.469+0000 7fe289600640 1 -- 172.21.15.47:0/1967708990 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c0cef30 con 0x7fe27c0dd600 2024-04-24T22:39:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.469+0000 7fe289600640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0dd600 0x7fe27c0ce7f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe2840740e0 tx=0x7fe284068df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.469+0000 7f085f400640 1 --2- 172.21.15.47:0/3263870323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086015a960 0x7f0860156770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f084c0097a0 tx=0x7f084c02a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599626ad3b335925 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.470+0000 7fe280c00640 1 -- 172.21.15.47:0/1967708990 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28406d040 con 0x7fe27c0dd600 2024-04-24T22:39:46.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.470+0000 7fe280c00640 1 -- 172.21.15.47:0/1967708990 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe284073370 con 0x7fe27c0dd600 2024-04-24T22:39:46.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.470+0000 7f085ea00640 1 -- 172.21.15.47:0/3263870323 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c037070 con 0x7f086015a960 2024-04-24T22:39:46.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.470+0000 7fe280c00640 1 -- 172.21.15.47:0/1967708990 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284073720 con 0x7fe27c0dd600 2024-04-24T22:39:46.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.470+0000 7f085ea00640 1 -- 172.21.15.47:0/3263870323 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c032440 con 0x7f086015a960 2024-04-24T22:39:46.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.470+0000 7f8b4b400640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8b3806beb0 0x7f8b3806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.470+0000 7f085ea00640 1 -- 172.21.15.47:0/3263870323 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c037070 con 0x7f086015a960 2024-04-24T22:39:46.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.470+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27c0d9790 con 0x7fe27c0dd600 2024-04-24T22:39:46.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.470+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe27c0d9d50 con 0x7fe27c0dd600 2024-04-24T22:39:46.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.470+0000 7f8b4b400640 1 -- 172.21.15.47:0/1386299925 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8b4c08afb0 con 0x7f8b50006f30 2024-04-24T22:39:46.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f8b4b400640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] conn(0x7f8b380740b0 0x7f8b380764d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f8b56a00640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8b3806beb0 0x7f8b3806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.469+0000 7f6231600640 1 -- 172.21.15.47:0/3031054797 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f622001a020 con 0x7f622c037670 2024-04-24T22:39:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f8b4b400640 1 -- 172.21.15.47:0/1386299925 --> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b38076b80 con 0x7f8b380740b0 2024-04-24T22:39:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f8b4b400640 1 -- 172.21.15.47:0/1386299925 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8b4c092050 con 0x7f8b50006f30 2024-04-24T22:39:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f8b57e00640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] conn(0x7f8b380740b0 0x7f8b380764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f0866a00640 1 -- 172.21.15.47:0/3263870323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086015a960 msgr2=0x7f0860156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f0866a00640 1 --2- 172.21.15.47:0/3263870323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086015a960 0x7f0860156770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f084c0097a0 tx=0x7f084c02a230 comp rx=0 tx=0).stop 2024-04-24T22:39:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.472+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe260001030 con 0x7fe27c0dd600 2024-04-24T22:39:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f0866a00640 1 -- 172.21.15.47:0/3263870323 shutdown_connections 2024-04-24T22:39:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f0866a00640 1 --2- 172.21.15.47:0/3263870323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086015a960 0x7f0860156770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f0866a00640 1 --2- 172.21.15.47:0/3263870323 >> v2:172.21.15.113:3300/0 conn(0x7f0860150ab0 0x7f08600c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.471+0000 7f0866a00640 1 -- 172.21.15.47:0/3263870323 >> 172.21.15.47:0/3263870323 conn(0x7f086006e580 msgr2=0x7f086006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.472+0000 7f0954e00640 1 Processor -- start 2024-04-24T22:39:46.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.472+0000 7f0866a00640 1 -- 172.21.15.47:0/3263870323 shutdown_connections 2024-04-24T22:39:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.472+0000 7f8b56a00640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8b3806beb0 0x7f8b3806e370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b44034000 tx=0x7f8b4402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.472+0000 7f0954e00640 1 -- start start 2024-04-24T22:39:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.473+0000 7f8b57e00640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] conn(0x7f8b380740b0 0x7f8b380764d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.473+0000 7f6231600640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f621406c060 0x7f621406e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.473+0000 7f6231600640 1 -- 172.21.15.47:0/3031054797 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f622008a160 con 0x7f622c037670 2024-04-24T22:39:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.473+0000 7f6239400640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f621406c060 0x7f621406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.472+0000 7f0c8f200640 1 Processor -- start 2024-04-24T22:39:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.472+0000 7f0c8f200640 1 -- start start 2024-04-24T22:39:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.472+0000 7f0866a00640 1 -- 172.21.15.47:0/3263870323 wait complete. 2024-04-24T22:39:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.474+0000 7f0954e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500c1a00 0x7f09500c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.474+0000 7f0954e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f095015aa30 con 0x7f0950073670 2024-04-24T22:39:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.474+0000 7f0954e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f095015ad10 con 0x7f0950151780 2024-04-24T22:39:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.474+0000 7f0954e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f095015dc10 con 0x7f09500c1a00 2024-04-24T22:39:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.474+0000 7f094de00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500c1a00 0x7f09500c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.474+0000 7f094de00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500c1a00 0x7f09500c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:59038/0 (socket says 172.21.15.47:59038) 2024-04-24T22:39:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.474+0000 7f094de00640 1 -- 172.21.15.47:0/1689559743 learned_addr learned my addr 172.21.15.47:0/1689559743 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.474+0000 7f0866a00640 1 Processor -- start 2024-04-24T22:39:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.475+0000 7f0866a00640 1 -- start start 2024-04-24T22:39:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.475+0000 7f8b4b400640 1 -- 172.21.15.47:0/1386299925 <== osd.1 v2:172.21.15.47:6800/623040628 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b38076b80 con 0x7f8b380740b0 2024-04-24T22:39:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.473+0000 7f6231600640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] conn(0x7f62140742e0 0x7f6214076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.474+0000 7f6239400640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f621406c060 0x7f621406e520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6228006fd0 tx=0x7f6228008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.475+0000 7f094de00640 1 -- 172.21.15.47:0/1689559743 >> v1:172.21.15.113:6789/0 conn(0x7f0950151780 legacy=0x7f0950072f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:39:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.475+0000 7f094de00640 1 -- 172.21.15.47:0/1689559743 >> v1:172.21.15.47:6789/0 conn(0x7f0950073670 legacy=0x7f09501576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.475+0000 7f094de00640 1 -- 172.21.15.47:0/1689559743 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095015aff0 con 0x7f09500c1a00 2024-04-24T22:39:46.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.475+0000 7f6231600640 1 -- 172.21.15.47:0/3031054797 --> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6214076db0 con 0x7f62140742e0 2024-04-24T22:39:46.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.474+0000 7fe280c00640 1 -- 172.21.15.47:0/1967708990 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2840a6440 con 0x7fe27c0dd600 2024-04-24T22:39:46.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7fe280c00640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe26406beb0 0x7fe26406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7fe280c00640 1 -- 172.21.15.47:0/1967708990 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe284118a20 con 0x7fe27c0dd600 2024-04-24T22:39:46.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7fe280c00640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] conn(0x7fe2640740d0 0x7fe2640764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7fe280c00640 1 -- 172.21.15.47:0/1967708990 --> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe264076ba0 con 0x7fe2640740d0 2024-04-24T22:39:46.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7fe280c00640 1 -- 172.21.15.47:0/1967708990 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe284118dd0 con 0x7fe27c0dd600 2024-04-24T22:39:46.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7fe288c00640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe26406beb0 0x7fe26406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7f6231600640 1 -- 172.21.15.47:0/3031054797 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6220091050 con 0x7f622c037670 2024-04-24T22:39:46.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7f6239e00640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] conn(0x7f62140742e0 0x7f6214076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7f0866a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08600c7940 0x7f0860148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7f0866a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0860150ab0 0x7f086014a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7f0866a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086015a960 0x7f086014a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7f0c8f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 0x7f0c90070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f0c8f200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c90154b10 con 0x7f0c9015afe0 2024-04-24T22:39:46.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f0c8f200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c900c6c70 con 0x7f0c900707e0 2024-04-24T22:39:46.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f0c8f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0c900c6f30 con 0x7f0c90154730 2024-04-24T22:39:46.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f094de00640 1 --2- 172.21.15.47:0/1689559743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500c1a00 0x7f09500c8590 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0940013180 tx=0x7f0940032a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1da40779813db5ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7f21ab600640 1 Processor -- start 2024-04-24T22:39:46.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.476+0000 7f21ab600640 1 -- start start 2024-04-24T22:39:46.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7fe283e00640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] conn(0x7fe2640740d0 0x7fe2640764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f094ca00640 1 -- 172.21.15.47:0/1689559743 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094003b020 con 0x7f09500c1a00 2024-04-24T22:39:46.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f094ca00640 1 -- 172.21.15.47:0/1689559743 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09400114e0 con 0x7f09500c1a00 2024-04-24T22:39:46.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f094ca00640 1 -- 172.21.15.47:0/1689559743 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094003b020 con 0x7f09500c1a00 2024-04-24T22:39:46.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f085f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0860150ab0 0x7f086014a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f085f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0860150ab0 0x7f086014a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:59046/0 (socket says 172.21.15.47:59046) 2024-04-24T22:39:46.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f085fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08600c7940 0x7f0860148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f0c8de00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 0x7f0c90070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f0c8de00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 0x7f0c90070ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:59060/0 (socket says 172.21.15.47:59060) 2024-04-24T22:39:46.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f0c8de00640 1 -- 172.21.15.47:0/3968590337 learned_addr learned my addr 172.21.15.47:0/3968590337 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7fe3ce800640 1 Processor -- start 2024-04-24T22:39:46.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7fe3ce800640 1 -- start start 2024-04-24T22:39:46.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7fe288c00640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe26406beb0 0x7fe26406e370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe278034000 tx=0x7fe27802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f085fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08600c7940 0x7f0860148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53936/0 (socket says 172.21.15.47:53936) 2024-04-24T22:39:46.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.477+0000 7f0c8d400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0c900707e0 0x7f0c9015e3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:51740/0 (socket says 172.21.15.47:51740) 2024-04-24T22:39:46.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f6239e00640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] conn(0x7f62140742e0 0x7f6214076700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0866a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0860150e90 con 0x7f08600c7940 2024-04-24T22:39:46.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0864a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086015a960 0x7f086014a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0864a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086015a960 0x7f086014a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37746/0 (socket says 172.21.15.47:37746) 2024-04-24T22:39:46.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0864a00640 1 -- 172.21.15.47:0/1458447069 learned_addr learned my addr 172.21.15.47:0/1458447069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0c8de00640 1 -- 172.21.15.47:0/3968590337 >> v1:172.21.15.113:6789/0 conn(0x7f0c900707e0 legacy=0x7f0c9015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0860156cb0 con 0x7f086015a960 2024-04-24T22:39:46.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0c8de00640 1 -- 172.21.15.47:0/3968590337 >> v1:172.21.15.47:6789/0 conn(0x7f0c9015afe0 legacy=0x7f0c9006e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0c8de00640 1 -- 172.21.15.47:0/3968590337 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c900c7210 con 0x7f0c90154730 2024-04-24T22:39:46.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0860070970 con 0x7f0860150ab0 2024-04-24T22:39:46.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0954e00640 1 -- 172.21.15.47:0/1689559743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500c1a00 msgr2=0x7f09500c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.478+0000 7f0954e00640 1 --2- 172.21.15.47:0/1689559743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500c1a00 0x7f09500c8590 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0940013180 tx=0x7f0940032a20 comp rx=0 tx=0).stop 2024-04-24T22:39:46.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0954e00640 1 -- 172.21.15.47:0/1689559743 shutdown_connections 2024-04-24T22:39:46.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0954e00640 1 --2- 172.21.15.47:0/1689559743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09500c1a00 0x7f09500c8590 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0954e00640 1 -- 172.21.15.47:0/1689559743 >> 172.21.15.47:0/1689559743 conn(0x7f095006e580 msgr2=0x7f095006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0864a00640 1 -- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0860150ab0 msgr2=0x7f086014a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0954e00640 1 -- 172.21.15.47:0/1689559743 shutdown_connections 2024-04-24T22:39:46.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f6231600640 1 -- 172.21.15.47:0/3031054797 <== osd.2 v2:172.21.15.47:6816/4187599812 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6214076db0 con 0x7f62140742e0 2024-04-24T22:39:46.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7fe3ce800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3c806b160 0x7fe3c80ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0864a00640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0860150ab0 0x7f086014a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0864a00640 1 -- 172.21.15.47:0/1458447069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08600c7940 msgr2=0x7f0860148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0864a00640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08600c7940 0x7f0860148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0864a00640 1 -- 172.21.15.47:0/1458447069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08600c5ea0 con 0x7f086015a960 2024-04-24T22:39:46.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0864a00640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086015a960 0x7f086014a790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f085800c760 tx=0x7f085800cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7f0c8de00640 1 --2- 172.21.15.47:0/3968590337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 0x7f0c90070ea0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009e80 tx=0x7f0c8802a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d75948eaf027e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7fe3ce800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3c8158fc0 con 0x7fe3c806bc50 2024-04-24T22:39:46.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7fe3ce800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3c815ead0 con 0x7fe3c80b2a80 2024-04-24T22:39:46.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7fe3ce800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c815edb0 con 0x7fe3c806b160 2024-04-24T22:39:46.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.479+0000 7f0954e00640 1 -- 172.21.15.47:0/1689559743 wait complete. 2024-04-24T22:39:46.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7f0857e00640 1 -- 172.21.15.47:0/1458447069 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858003da0 con 0x7f086015a960 2024-04-24T22:39:46.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7f0857e00640 1 -- 172.21.15.47:0/1458447069 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0858005070 con 0x7f086015a960 2024-04-24T22:39:46.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7f0857e00640 1 -- 172.21.15.47:0/1458447069 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858005460 con 0x7f086015a960 2024-04-24T22:39:46.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08601eb8c0 con 0x7f086015a960 2024-04-24T22:39:46.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7f0c8ca00640 1 -- 172.21.15.47:0/3968590337 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88036070 con 0x7f0c90154730 2024-04-24T22:39:46.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.481+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08601ebe80 con 0x7f086015a960 2024-04-24T22:39:46.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.481+0000 7fe283e00640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] conn(0x7fe2640740d0 0x7fe2640764f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7fe3c7400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe3c806b160 0x7fe3c80ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7fe3c6a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe3c80b2a80 0x7fe3c806b540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:51750/0 (socket says 172.21.15.47:51750) 2024-04-24T22:39:46.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7fe3c6a00640 1 -- 172.21.15.47:0/2334609987 learned_addr learned my addr 172.21.15.47:0/2334609987 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:46.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.481+0000 7fe3c6000640 1 -- 172.21.15.47:0/2334609987 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1535286345 0 0) 0x7fe3c815ead0 con 0x7fe3c80b2a80 2024-04-24T22:39:46.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7f21ab600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f21a406b160 0x7f21a40ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7f0c8ca00640 1 -- 172.21.15.47:0/3968590337 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c88031600 con 0x7f0c90154730 2024-04-24T22:39:46.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7f0c8ca00640 1 -- 172.21.15.47:0/3968590337 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c880319d0 con 0x7f0c90154730 2024-04-24T22:39:46.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.481+0000 7f0c8f200640 1 -- 172.21.15.47:0/3968590337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 msgr2=0x7f0c90070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.481+0000 7f0c8f200640 1 --2- 172.21.15.47:0/3968590337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 0x7f0c90070ea0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009e80 tx=0x7f0c8802a7a0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.482+0000 7f0954e00640 1 Processor -- start 2024-04-24T22:39:46.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.480+0000 7f21ab600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 0x7f21a406b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.481+0000 7f21ab600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a4158fc0 con 0x7f21a406ba80 2024-04-24T22:39:46.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.481+0000 7f21ab600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f21a406bf10 con 0x7f21a406b160 2024-04-24T22:39:46.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.481+0000 7f21ab600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f21a415ead0 con 0x7f21a40b2a80 2024-04-24T22:39:46.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.481+0000 7fe3c6000640 1 -- 172.21.15.47:0/2334609987 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3ac003520 con 0x7fe3c80b2a80 2024-04-24T22:39:46.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.481+0000 7fe3c7400640 1 -- 172.21.15.47:0/2334609987 >> v1:172.21.15.113:6789/0 conn(0x7fe3c80b2a80 legacy=0x7fe3c806b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.482+0000 7fe3c7400640 1 -- 172.21.15.47:0/2334609987 >> v1:172.21.15.47:6789/0 conn(0x7fe3c806bc50 legacy=0x7fe3c80cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.482+0000 7fe3c7400640 1 -- 172.21.15.47:0/2334609987 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c815f090 con 0x7fe3c806b160 2024-04-24T22:39:46.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.482+0000 7f0c8f200640 1 -- 172.21.15.47:0/3968590337 shutdown_connections 2024-04-24T22:39:46.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.482+0000 7f0c8f200640 1 --2- 172.21.15.47:0/3968590337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 0x7f0c90070ea0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.482+0000 7f0c8f200640 1 -- 172.21.15.47:0/3968590337 >> 172.21.15.47:0/3968590337 conn(0x7f0c900cb900 msgr2=0x7f0c900cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.482+0000 7f0c8f200640 1 -- 172.21.15.47:0/3968590337 shutdown_connections 2024-04-24T22:39:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.482+0000 7f0c8f200640 1 -- 172.21.15.47:0/3968590337 wait complete. 2024-04-24T22:39:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.482+0000 7f0954e00640 1 -- start start 2024-04-24T22:39:46.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.483+0000 7f085f400640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0860150ab0 0x7f086014a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:39:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.483+0000 7fe280c00640 1 -- 172.21.15.47:0/1967708990 <== osd.0 v2:172.21.15.47:6808/2543493744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe264076ba0 con 0x7fe2640740d0 2024-04-24T22:39:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.483+0000 7f21a3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 0x7f21a406b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.483+0000 7f21a3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 0x7f21a406b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:59070/0 (socket says 172.21.15.47:59070) 2024-04-24T22:39:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.483+0000 7f21a3e00640 1 -- 172.21.15.47:0/3217352570 learned_addr learned my addr 172.21.15.47:0/3217352570 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.483+0000 7f21a8c00640 1 --2- 172.21.15.47:0/3217352570 >> v2:172.21.15.47:3300/0 conn(0x7f21a406b160 0x7f21a40ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.483+0000 7fe3c7400640 1 --2- 172.21.15.47:0/2334609987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c806b160 0x7fe3c80ca7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009a00 tx=0x7fe3b402a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405ee233292ec45e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.484+0000 7fe3c6000640 1 -- 172.21.15.47:0/2334609987 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4036070 con 0x7fe3c806b160 2024-04-24T22:39:46.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.484+0000 7f21a3400640 1 -- 172.21.15.47:0/3217352570 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 166948106 0 0) 0x7f21a4158fc0 con 0x7f21a406ba80 2024-04-24T22:39:46.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7fe3c6000640 1 -- 172.21.15.47:0/2334609987 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b4031600 con 0x7fe3c806b160 2024-04-24T22:39:46.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7f21a3400640 1 -- 172.21.15.47:0/3217352570 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2188003520 con 0x7f21a406ba80 2024-04-24T22:39:46.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.484+0000 7f0c8f200640 1 Processor -- start 2024-04-24T22:39:46.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7f0c8f200640 1 -- start start 2024-04-24T22:39:46.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0838001030 con 0x7f086015a960 2024-04-24T22:39:46.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7f0c8f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c900707e0 0x7f0c90145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7fe3c6000640 1 -- 172.21.15.47:0/2334609987 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4036070 con 0x7fe3c806b160 2024-04-24T22:39:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7fe3ce800640 1 -- 172.21.15.47:0/2334609987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c806b160 msgr2=0x7fe3c80ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7fe3ce800640 1 --2- 172.21.15.47:0/2334609987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c806b160 0x7fe3c80ca7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009a00 tx=0x7fe3b402a430 comp rx=0 tx=0).stop 2024-04-24T22:39:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7fe3ce800640 1 -- 172.21.15.47:0/2334609987 shutdown_connections 2024-04-24T22:39:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7fe3ce800640 1 --2- 172.21.15.47:0/2334609987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c806b160 0x7fe3c80ca7d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7fe3ce800640 1 -- 172.21.15.47:0/2334609987 >> 172.21.15.47:0/2334609987 conn(0x7fe3c806c940 msgr2=0x7fe3c8072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7fe3ce800640 1 -- 172.21.15.47:0/2334609987 shutdown_connections 2024-04-24T22:39:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7f0954e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0950073670 0x7f0950138610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7f0c8f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 0x7f0c901523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7f21a8c00640 1 -- 172.21.15.47:0/3217352570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 msgr2=0x7f21a406b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7f21a8c00640 1 --2- 172.21.15.47:0/3217352570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 0x7f21a406b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7f21a8c00640 1 -- 172.21.15.47:0/3217352570 >> v1:172.21.15.113:6789/0 conn(0x7f21a406ba80 legacy=0x7f21a40cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7f21a8c00640 1 -- 172.21.15.47:0/3217352570 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a415edb0 con 0x7f21a406b160 2024-04-24T22:39:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.485+0000 7f21a3e00640 1 --2- 172.21.15.47:0/3217352570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 0x7f21a406b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7f0954e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0950151780 0x7f0950071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7f0954e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f095015b7f0 0x7f0950071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7f0954e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0950157e00 con 0x7f0950073670 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7f0954e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f095015b5b0 con 0x7f095015b7f0 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.486+0000 7f0954e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0950157f80 con 0x7f0950151780 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f094d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0950151780 0x7f0950071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f094de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0950073670 0x7f0950138610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f094e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f095015b7f0 0x7f0950071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f094e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f095015b7f0 0x7f0950071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37754/0 (socket says 172.21.15.47:37754) 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f094de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0950073670 0x7f0950138610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53970/0 (socket says 172.21.15.47:53970) 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f094e800640 1 -- 172.21.15.47:0/2623510365 learned_addr learned my addr 172.21.15.47:0/2623510365 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f21a8c00640 1 --2- 172.21.15.47:0/3217352570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a406b160 0x7f21a40ca7d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21940097a0 tx=0x7f219402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59f4a4ee31563dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f0c8f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c9015afe0 0x7f0c90152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f0c8f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9015aac0 con 0x7f0c900707e0 2024-04-24T22:39:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f0c8f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9015a7e0 con 0x7f0c9015afe0 2024-04-24T22:39:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f0c8f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0c900c7750 con 0x7f0c90154730 2024-04-24T22:39:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f0c8e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c9015afe0 0x7f0c90152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7fe3ce800640 1 -- 172.21.15.47:0/2334609987 wait complete. 2024-04-24T22:39:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f0c8e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c9015afe0 0x7f0c90152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37770/0 (socket says 172.21.15.47:37770) 2024-04-24T22:39:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f0c8e800640 1 -- 172.21.15.47:0/486532204 learned_addr learned my addr 172.21.15.47:0/486532204 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f0c8de00640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c900707e0 0x7f0c90145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f0c8d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 0x7f0c901523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f0857e00640 1 -- 172.21.15.47:0/1458447069 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0858005600 con 0x7f086015a960 2024-04-24T22:39:46.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f21a3400640 1 -- 172.21.15.47:0/3217352570 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194037070 con 0x7f21a406b160 2024-04-24T22:39:46.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f21a3400640 1 -- 172.21.15.47:0/3217352570 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2194032440 con 0x7f21a406b160 2024-04-24T22:39:46.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f0c8e800640 1 -- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 msgr2=0x7f0c901523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f0c8e800640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c90154730 0x7f0c901523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f0c8e800640 1 -- 172.21.15.47:0/486532204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c900707e0 msgr2=0x7f0c90145070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f0c8e800640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c900707e0 0x7f0c90145070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f0c8e800640 1 -- 172.21.15.47:0/486532204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c90152000 con 0x7f0c9015afe0 2024-04-24T22:39:46.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f0c8de00640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c900707e0 0x7f0c90145070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:46.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.488+0000 7f21a3400640 1 -- 172.21.15.47:0/3217352570 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194032850 con 0x7f21a406b160 2024-04-24T22:39:46.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f094e800640 1 -- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0950151780 msgr2=0x7f0950071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f094e800640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0950151780 0x7f0950071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.487+0000 7f094e800640 1 -- 172.21.15.47:0/2623510365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0950073670 msgr2=0x7f0950138610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f0c8e800640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c9015afe0 0x7f0c90152930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c3f0 tx=0x7f0c8000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f094e800640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0950073670 0x7f0950138610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f094e800640 1 -- 172.21.15.47:0/2623510365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095014f010 con 0x7f095015b7f0 2024-04-24T22:39:46.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f094d400640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0950151780 0x7f0950071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:46.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f0c86000640 1 -- 172.21.15.47:0/486532204 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800052d0 con 0x7f0c9015afe0 2024-04-24T22:39:46.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f0c86000640 1 -- 172.21.15.47:0/486532204 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c80005470 con 0x7f0c9015afe0 2024-04-24T22:39:46.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f0c86000640 1 -- 172.21.15.47:0/486532204 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80007260 con 0x7f0c9015afe0 2024-04-24T22:39:46.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f21ab600640 1 -- 172.21.15.47:0/3217352570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a406b160 msgr2=0x7f21a40ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f21ab600640 1 --2- 172.21.15.47:0/3217352570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a406b160 0x7f21a40ca7d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21940097a0 tx=0x7f219402a1a0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c90151d20 con 0x7f0c9015afe0 2024-04-24T22:39:46.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c901e5f50 con 0x7f0c9015afe0 2024-04-24T22:39:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.489+0000 7f094e800640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f095015b7f0 0x7f0950071590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f094800c730 tx=0x7f094800cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7f21ab600640 1 -- 172.21.15.47:0/3217352570 shutdown_connections 2024-04-24T22:39:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7f21ab600640 1 --2- 172.21.15.47:0/3217352570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 0x7f21a406b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7f21ab600640 1 --2- 172.21.15.47:0/3217352570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a406b160 0x7f21a40ca7d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7f21ab600640 1 -- 172.21.15.47:0/3217352570 >> 172.21.15.47:0/3217352570 conn(0x7f21a406c940 msgr2=0x7f21a4072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7fe3ce800640 1 Processor -- start 2024-04-24T22:39:46.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7f21ab600640 1 -- 172.21.15.47:0/3217352570 shutdown_connections 2024-04-24T22:39:46.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7f21ab600640 1 -- 172.21.15.47:0/3217352570 wait complete. 2024-04-24T22:39:46.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7fe3ce800640 1 -- start start 2024-04-24T22:39:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7f0946000640 1 -- 172.21.15.47:0/2623510365 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948016020 con 0x7f095015b7f0 2024-04-24T22:39:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7f0946000640 1 -- 172.21.15.47:0/2623510365 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0948006d70 con 0x7f095015b7f0 2024-04-24T22:39:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7f0946000640 1 -- 172.21.15.47:0/2623510365 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948003bb0 con 0x7f095015b7f0 2024-04-24T22:39:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.490+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09500c9d40 con 0x7f095015b7f0 2024-04-24T22:39:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.491+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095015d270 con 0x7f095015b7f0 2024-04-24T22:39:46.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.491+0000 7f0857e00640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f084006bf00 0x7f084006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.491+0000 7fe3ce800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3c806b160 0x7fe3c8146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.491+0000 7f085fe00640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f084006bf00 0x7f084006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.491+0000 7f0857e00640 1 -- 172.21.15.47:0/1458447069 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f085808ae40 con 0x7f086015a960 2024-04-24T22:39:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.491+0000 7fe3ce800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c806bc50 0x7fe3c80b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.492+0000 7f0857e00640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] conn(0x7f0840074180 0x7f08400765a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.492+0000 7f0857e00640 1 -- 172.21.15.47:0/1458447069 --> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0840076c50 con 0x7f0840074180 2024-04-24T22:39:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.492+0000 7f0857e00640 1 -- 172.21.15.47:0/1458447069 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f085808d4a0 con 0x7f086015a960 2024-04-24T22:39:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.491+0000 7fe3ce800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c80b2a80 0x7fe3c81f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.491+0000 7fe3ce800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c80ccb00 con 0x7fe3c806bc50 2024-04-24T22:39:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.491+0000 7fe3ce800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c815f650 con 0x7fe3c80b2a80 2024-04-24T22:39:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.491+0000 7fe3ce800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c80ccc80 con 0x7fe3c806b160 2024-04-24T22:39:46.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.492+0000 7f085fe00640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f084006bf00 0x7f084006e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0850009b70 tx=0x7f0850008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.492+0000 7fe3c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c80b2a80 0x7fe3c81f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.492+0000 7fe3c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c80b2a80 0x7fe3c81f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37784/0 (socket says 172.21.15.47:37784) 2024-04-24T22:39:46.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.492+0000 7fe3c7e00640 1 -- 172.21.15.47:0/2871598351 learned_addr learned my addr 172.21.15.47:0/2871598351 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.492+0000 7fe3c7400640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3c806b160 0x7fe3c8146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.492+0000 7f094de00640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0950073670 0x7f0950138610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:46.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.493+0000 7fe3c6a00640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c806bc50 0x7fe3c80b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.493+0000 7f0946000640 1 -- 172.21.15.47:0/2623510365 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09480050d0 con 0x7f095015b7f0 2024-04-24T22:39:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.492+0000 7f0c86000640 1 -- 172.21.15.47:0/486532204 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c80007400 con 0x7f0c9015afe0 2024-04-24T22:39:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.493+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c90068d50 con 0x7f0c9015afe0 2024-04-24T22:39:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.493+0000 7f0c86000640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0c7006bf00 0x7f0c7006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.493+0000 7f0c86000640 1 -- 172.21.15.47:0/486532204 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0c8008a160 con 0x7f0c9015afe0 2024-04-24T22:39:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7f0c8de00640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0c7006bf00 0x7f0c7006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7f0c86000640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] conn(0x7f0c70074180 0x7f0c700765a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7f0c86000640 1 -- 172.21.15.47:0/486532204 --> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c80009e50 con 0x7f0c70074180 2024-04-24T22:39:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7f0c86000640 1 -- 172.21.15.47:0/486532204 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0c80091050 con 0x7f0c9015afe0 2024-04-24T22:39:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7f0c8d400640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] conn(0x7f0c70074180 0x7f0c700765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7f085f400640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] conn(0x7f0840074180 0x7f08400765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.495+0000 7f21ab600640 1 Processor -- start 2024-04-24T22:39:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.495+0000 7f0c8de00640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0c7006bf00 0x7f0c7006e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0c88033000 tx=0x7f0c88009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.495+0000 7f0c8d400640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] conn(0x7f0c70074180 0x7f0c700765a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.495+0000 7f085f400640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] conn(0x7f0840074180 0x7f08400765a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.495+0000 7f21ab600640 1 -- start start 2024-04-24T22:39:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7fe3c6a00640 1 -- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3c806b160 msgr2=0x7fe3c8146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7fe3c6a00640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3c806b160 0x7fe3c8146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7fe3c6a00640 1 -- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c80b2a80 msgr2=0x7fe3c81f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7fe3c6a00640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3c80b2a80 0x7fe3c81f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.494+0000 7fe3c6a00640 1 -- 172.21.15.47:0/2871598351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c80b3260 con 0x7fe3c806bc50 2024-04-24T22:39:46.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.495+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0950073b90 con 0x7f095015b7f0 2024-04-24T22:39:46.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.495+0000 7f0946000640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f093006be80 0x7f093006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.495+0000 7f0946000640 1 -- 172.21.15.47:0/2623510365 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f094808ac80 con 0x7f095015b7f0 2024-04-24T22:39:46.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.496+0000 7f0946000640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] conn(0x7f0930074080 0x7f09300764a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.496+0000 7f0946000640 1 -- 172.21.15.47:0/2623510365 --> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f094801eec0 con 0x7f0930074080 2024-04-24T22:39:46.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.496+0000 7f0946000640 1 -- 172.21.15.47:0/2623510365 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0948091020 con 0x7f095015b7f0 2024-04-24T22:39:46.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.496+0000 7fe3c6a00640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c806bc50 0x7fe3c80b4bc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800c3c0 tx=0x7fe3b8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.496+0000 7f21ab600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a406ba80 0x7f21a41f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.496+0000 7f21ab600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 0x7f21a40cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7f0c86000640 1 -- 172.21.15.47:0/486532204 <== osd.4 v2:172.21.15.113:6817/2737975939 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c80009e50 con 0x7f0c70074180 2024-04-24T22:39:46.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.496+0000 7f21ab600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a415d710 0x7f21a41f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.496+0000 7f21ab600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f21a415f650 con 0x7f21a415d710 2024-04-24T22:39:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.496+0000 7f21ab600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f21a415f370 con 0x7f21a406ba80 2024-04-24T22:39:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.496+0000 7f21ab600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f21a40ccb00 con 0x7f21a40b2a80 2024-04-24T22:39:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7f21a8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a406ba80 0x7f21a41f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7fe3bf400640 1 -- 172.21.15.47:0/2871598351 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8017070 con 0x7fe3c806bc50 2024-04-24T22:39:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7fe3bf400640 1 -- 172.21.15.47:0/2871598351 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b8005ce0 con 0x7fe3c806bc50 2024-04-24T22:39:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7fe3bf400640 1 -- 172.21.15.47:0/2871598351 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b80072d0 con 0x7fe3c806bc50 2024-04-24T22:39:46.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7f094d400640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] conn(0x7f0930074080 0x7f09300764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7f094de00640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f093006be80 0x7f093006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7f094d400640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] conn(0x7f0930074080 0x7f09300764a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7f21a9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a415d710 0x7f21a41f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7f21a8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a406ba80 0x7f21a41f89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37790/0 (socket says 172.21.15.47:37790) 2024-04-24T22:39:46.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7f21a8c00640 1 -- 172.21.15.47:0/1629204698 learned_addr learned my addr 172.21.15.47:0/1629204698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.498+0000 7f21a3e00640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 0x7f21a40cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.498+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c81f4570 con 0x7fe3c806bc50 2024-04-24T22:39:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.498+0000 7f21a8c00640 1 -- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 msgr2=0x7f21a40cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.498+0000 7f21a8c00640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40b2a80 0x7f21a40cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.498+0000 7f21a8c00640 1 -- 172.21.15.47:0/1629204698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a415d710 msgr2=0x7f21a41f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.497+0000 7f0857e00640 1 -- 172.21.15.47:0/1458447069 <== osd.3 v2:172.21.15.47:6824/872127764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0840076c50 con 0x7f0840074180 2024-04-24T22:39:46.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.498+0000 7f094de00640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f093006be80 0x7f093006e340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f094003c000 tx=0x7f0940044040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.498+0000 7f0946000640 1 -- 172.21.15.47:0/2623510365 <== osd.5 v2:172.21.15.113:6800/1697310705 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f094801eec0 con 0x7f0930074080 2024-04-24T22:39:46.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.498+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c81611f0 con 0x7fe3c806bc50 2024-04-24T22:39:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.499+0000 7f21a8c00640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a415d710 0x7f21a41f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.499+0000 7f21a8c00640 1 -- 172.21.15.47:0/1629204698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a415de50 con 0x7f21a406ba80 2024-04-24T22:39:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.499+0000 7f21a9600640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21a415d710 0x7f21a41f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:39:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.499+0000 7f21a8c00640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a406ba80 0x7f21a41f89e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2194009890 tx=0x7f21940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.499+0000 7f21a0c00640 1 -- 172.21.15.47:0/1629204698 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194037050 con 0x7f21a406ba80 2024-04-24T22:39:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.499+0000 7f21a0c00640 1 -- 172.21.15.47:0/1629204698 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2194038040 con 0x7f21a406ba80 2024-04-24T22:39:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.499+0000 7f21a0c00640 1 -- 172.21.15.47:0/1629204698 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219403ca70 con 0x7f21a406ba80 2024-04-24T22:39:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.499+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a415e110 con 0x7f21a406ba80 2024-04-24T22:39:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.499+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a41f5090 con 0x7f21a406ba80 2024-04-24T22:39:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.501+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21a4073b90 con 0x7f21a406ba80 2024-04-24T22:39:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.502+0000 7f21a0c00640 1 -- 172.21.15.47:0/1629204698 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f219403cc10 con 0x7f21a406ba80 2024-04-24T22:39:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.503+0000 7f21a0c00640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f218406be30 0x7f218406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.503+0000 7f21a0c00640 1 -- 172.21.15.47:0/1629204698 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f21940ac330 con 0x7f21a406ba80 2024-04-24T22:39:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.500+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe3c8073b90 con 0x7fe3c806bc50 2024-04-24T22:39:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.503+0000 7f21a0c00640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] conn(0x7f2184074030 0x7f2184076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.503+0000 7f21a0c00640 1 -- 172.21.15.47:0/1629204698 --> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2194007e60 con 0x7f2184074030 2024-04-24T22:39:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.503+0000 7f21a0c00640 1 -- 172.21.15.47:0/1629204698 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f21940ac6e0 con 0x7f21a406ba80 2024-04-24T22:39:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.503+0000 7f21a3e00640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f218406be30 0x7f218406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.503+0000 7f21a9600640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] conn(0x7f2184074030 0x7f2184076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.504+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 --> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f622c039640 con 0x7f62140742e0 2024-04-24T22:39:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.504+0000 7f21a3e00640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f218406be30 0x7f218406e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f21a40ca1e0 tx=0x7f2190008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.504+0000 7f21a9600640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] conn(0x7f2184074030 0x7f2184076450 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.504+0000 7f6231600640 1 -- 172.21.15.47:0/3031054797 <== osd.2 v2:172.21.15.47:6816/4187599812 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f622c039640 con 0x7f62140742e0 2024-04-24T22:39:46.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.505+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 >> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] conn(0x7f62140742e0 msgr2=0x7f6214076700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.505+0000 7f623a800640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] conn(0x7f62140742e0 0x7f6214076700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.505+0000 7f21a0c00640 1 -- 172.21.15.47:0/1629204698 <== osd.6 v2:172.21.15.113:6808/1023357201 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2194007e60 con 0x7f2184074030 2024-04-24T22:39:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.507+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f621406c060 msgr2=0x7f621406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.507+0000 7f623a800640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f621406c060 0x7f621406e520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6228006fd0 tx=0x7f6228008040 comp rx=0 tx=0).stop 2024-04-24T22:39:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.507+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f622c037670 msgr2=0x7f622c0ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.507+0000 7f623a800640 1 --2- 172.21.15.47:0/3031054797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f622c037670 0x7f622c0ca880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f622000c3e0 tx=0x7f622000c8a0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.507+0000 7f6239e00640 1 -- 172.21.15.47:0/3031054797 reap_dead start 2024-04-24T22:39:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.507+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 shutdown_connections 2024-04-24T22:39:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.507+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 >> 172.21.15.47:0/3031054797 conn(0x7f622c054610 msgr2=0x7f622c050340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.511+0000 7fe3bf400640 1 -- 172.21.15.47:0/2871598351 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3b801a070 con 0x7fe3c806bc50 2024-04-24T22:39:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.512+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 shutdown_connections 2024-04-24T22:39:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.512+0000 7f623a800640 1 -- 172.21.15.47:0/3031054797 wait complete. 2024-04-24T22:39:46.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.513+0000 7fe3bf400640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe3a806bf00 0x7fe3a806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.513+0000 7fe3bf400640 1 -- 172.21.15.47:0/2871598351 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe3b808a0c0 con 0x7fe3c806bc50 2024-04-24T22:39:46.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.513+0000 7fe3c7400640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe3a806bf00 0x7fe3a806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.513+0000 7fe3bf400640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] conn(0x7fe3a8074180 0x7fe3a80765a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.513+0000 7fe3bf400640 1 -- 172.21.15.47:0/2871598351 --> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3b8010e50 con 0x7fe3a8074180 2024-04-24T22:39:46.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.514+0000 7fe3bf400640 1 -- 172.21.15.47:0/2871598351 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe3b8091050 con 0x7fe3c806bc50 2024-04-24T22:39:46.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.514+0000 7fe3c7e00640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] conn(0x7fe3a8074180 0x7fe3a80765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.514+0000 7fe3c7400640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe3a806bf00 0x7fe3a806e3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4033000 tx=0x7fe3b4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.515+0000 7fe3c7e00640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] conn(0x7fe3a8074180 0x7fe3a80765a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.516+0000 7fe3bf400640 1 -- 172.21.15.47:0/2871598351 <== osd.7 v2:172.21.15.113:6816/2022093233 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3b8010e50 con 0x7fe3a8074180 2024-04-24T22:39:46.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.522+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 --> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe260002dd0 con 0x7fe2640740d0 2024-04-24T22:39:46.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.523+0000 7fe280c00640 1 -- 172.21.15.47:0/1967708990 <== osd.0 v2:172.21.15.47:6808/2543493744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe260002dd0 con 0x7fe2640740d0 2024-04-24T22:39:46.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.524+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 >> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] conn(0x7fe2640740d0 msgr2=0x7fe2640764f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.524+0000 7fe28a000640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] conn(0x7fe2640740d0 0x7fe2640764f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.524+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe26406beb0 msgr2=0x7fe26406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.524+0000 7fe28a000640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe26406beb0 0x7fe26406e370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe278034000 tx=0x7fe27802e000 comp rx=0 tx=0).stop 2024-04-24T22:39:46.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.524+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0dd600 msgr2=0x7fe27c0ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.524+0000 7fe28a000640 1 --2- 172.21.15.47:0/1967708990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27c0dd600 0x7fe27c0ce7f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe2840740e0 tx=0x7fe284068df0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.524+0000 7fe289600640 1 -- 172.21.15.47:0/1967708990 reap_dead start 2024-04-24T22:39:46.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.526+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 shutdown_connections 2024-04-24T22:39:46.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.526+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 >> 172.21.15.47:0/1967708990 conn(0x7fe27c00ee70 msgr2=0x7fe27c0389c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.526+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 --> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b5003a020 con 0x7f8b380740b0 2024-04-24T22:39:46.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.527+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 shutdown_connections 2024-04-24T22:39:46.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.527+0000 7f8b4b400640 1 -- 172.21.15.47:0/1386299925 <== osd.1 v2:172.21.15.47:6800/623040628 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8b5003a020 con 0x7f8b380740b0 2024-04-24T22:39:46.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.527+0000 7fe28a000640 1 -- 172.21.15.47:0/1967708990 wait complete. 2024-04-24T22:39:46.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.528+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 >> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] conn(0x7f8b380740b0 msgr2=0x7f8b380764d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.528+0000 7f8b5cc00640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] conn(0x7f8b380740b0 0x7f8b380764d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.528+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 --> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0838002dd0 con 0x7f0840074180 2024-04-24T22:39:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.528+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8b3806beb0 msgr2=0x7f8b3806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.528+0000 7f8b5cc00640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8b3806beb0 0x7f8b3806e370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b44034000 tx=0x7f8b4402e000 comp rx=0 tx=0).stop 2024-04-24T22:39:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.528+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b50006f30 msgr2=0x7f8b500df740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.528+0000 7f8b5cc00640 1 --2- 172.21.15.47:0/1386299925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b50006f30 0x7f8b500df740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00ec40 tx=0x7f8b4c00c520 comp rx=0 tx=0).stop 2024-04-24T22:39:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.529+0000 7f8b57e00640 1 -- 172.21.15.47:0/1386299925 reap_dead start 2024-04-24T22:39:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.529+0000 7f0857e00640 1 -- 172.21.15.47:0/1458447069 <== osd.3 v2:172.21.15.47:6824/872127764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0838002dd0 con 0x7f0840074180 2024-04-24T22:39:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.530+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 >> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] conn(0x7f0840074180 msgr2=0x7f08400765a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.530+0000 7f0866a00640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] conn(0x7f0840074180 0x7f08400765a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.529+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 shutdown_connections 2024-04-24T22:39:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.529+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 >> 172.21.15.47:0/1386299925 conn(0x7f8b500061a0 msgr2=0x7f8b500e1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 shutdown_connections 2024-04-24T22:39:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f084006bf00 msgr2=0x7f084006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f8b5cc00640 1 -- 172.21.15.47:0/1386299925 wait complete. 2024-04-24T22:39:46.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f0866a00640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f084006bf00 0x7f084006e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0850009b70 tx=0x7f0850008040 comp rx=0 tx=0).stop 2024-04-24T22:39:46.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086015a960 msgr2=0x7f086014a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f0866a00640 1 --2- 172.21.15.47:0/1458447069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f086015a960 0x7f086014a790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f085800c760 tx=0x7f085800cc20 comp rx=0 tx=0).stop 2024-04-24T22:39:46.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f0864a00640 1 -- 172.21.15.47:0/1458447069 reap_dead start 2024-04-24T22:39:46.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 shutdown_connections 2024-04-24T22:39:46.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 >> 172.21.15.47:0/1458447069 conn(0x7f086006e580 msgr2=0x7f086006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 shutdown_connections 2024-04-24T22:39:46.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.531+0000 7f0866a00640 1 -- 172.21.15.47:0/1458447069 wait complete. 2024-04-24T22:39:46.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.532+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 --> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c901521d0 con 0x7f0c70074180 2024-04-24T22:39:46.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.532+0000 7f0c86000640 1 -- 172.21.15.47:0/486532204 <== osd.4 v2:172.21.15.113:6817/2737975939 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0c901521d0 con 0x7f0c70074180 2024-04-24T22:39:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.533+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] conn(0x7f0c70074180 msgr2=0x7f0c700765a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.533+0000 7f0c8f200640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] conn(0x7f0c70074180 0x7f0c700765a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.533+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0c7006bf00 msgr2=0x7f0c7006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.533+0000 7f0c8f200640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0c7006bf00 0x7f0c7006e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0c88033000 tx=0x7f0c88009000 comp rx=0 tx=0).stop 2024-04-24T22:39:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.533+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c9015afe0 msgr2=0x7f0c90152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.533+0000 7f0c8f200640 1 --2- 172.21.15.47:0/486532204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c9015afe0 0x7f0c90152930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c3f0 tx=0x7f0c8000c8b0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.533+0000 7f0c8e800640 1 -- 172.21.15.47:0/486532204 reap_dead start 2024-04-24T22:39:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.533+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 shutdown_connections 2024-04-24T22:39:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.533+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 >> 172.21.15.47:0/486532204 conn(0x7f0c900cb900 msgr2=0x7f0c900c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.574 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-04-24T22:39:46.575 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T22:39:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.537+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 shutdown_connections 2024-04-24T22:39:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.537+0000 7f0c8f200640 1 -- 172.21.15.47:0/486532204 wait complete. 2024-04-24T22:39:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.537+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 --> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0950141400 con 0x7f0930074080 2024-04-24T22:39:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.538+0000 7f0946000640 1 -- 172.21.15.47:0/2623510365 <== osd.5 v2:172.21.15.113:6800/1697310705 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0950141400 con 0x7f0930074080 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.539+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] conn(0x7f0930074080 msgr2=0x7f09300764a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.539+0000 7f0954e00640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] conn(0x7f0930074080 0x7f09300764a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.539+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f093006be80 msgr2=0x7f093006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.539+0000 7f0954e00640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f093006be80 0x7f093006e340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f094003c000 tx=0x7f0940044040 comp rx=0 tx=0).stop 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.539+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f095015b7f0 msgr2=0x7f0950071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.539+0000 7f0954e00640 1 --2- 172.21.15.47:0/2623510365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f095015b7f0 0x7f0950071590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f094800c730 tx=0x7f094800cbf0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.539+0000 7f094e800640 1 -- 172.21.15.47:0/2623510365 reap_dead start 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.540+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 shutdown_connections 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.540+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 >> 172.21.15.47:0/2623510365 conn(0x7f095006e580 msgr2=0x7f095013d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.540+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 shutdown_connections 2024-04-24T22:39:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.540+0000 7f0954e00640 1 -- 172.21.15.47:0/2623510365 wait complete. 2024-04-24T22:39:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.541+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 --> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21a40cadb0 con 0x7f2184074030 2024-04-24T22:39:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.542+0000 7f21a0c00640 1 -- 172.21.15.47:0/1629204698 <== osd.6 v2:172.21.15.113:6808/1023357201 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21a40cadb0 con 0x7f2184074030 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.543+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] conn(0x7f2184074030 msgr2=0x7f2184076450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.543+0000 7f21ab600640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] conn(0x7f2184074030 0x7f2184076450 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.544+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f218406be30 msgr2=0x7f218406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.544+0000 7f21ab600640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f218406be30 0x7f218406e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f21a40ca1e0 tx=0x7f2190008040 comp rx=0 tx=0).stop 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.544+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a406ba80 msgr2=0x7f21a41f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.544+0000 7f21ab600640 1 --2- 172.21.15.47:0/1629204698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a406ba80 0x7f21a41f89e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2194009890 tx=0x7f21940046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.544+0000 7f21a9600640 1 -- 172.21.15.47:0/1629204698 reap_dead start 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.544+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 shutdown_connections 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.544+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 >> 172.21.15.47:0/1629204698 conn(0x7f21a406c940 msgr2=0x7f21a40cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.544+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 shutdown_connections 2024-04-24T22:39:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.545+0000 7f21ab600640 1 -- 172.21.15.47:0/1629204698 wait complete. 2024-04-24T22:39:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.547+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 --> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe3c81f4b10 con 0x7fe3a8074180 2024-04-24T22:39:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.547+0000 7fe3bf400640 1 -- 172.21.15.47:0/2871598351 <== osd.7 v2:172.21.15.113:6816/2022093233 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe3c81f4b10 con 0x7fe3a8074180 2024-04-24T22:39:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.548+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] conn(0x7fe3a8074180 msgr2=0x7fe3a80765a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.548+0000 7fe3ce800640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] conn(0x7fe3a8074180 0x7fe3a80765a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.548+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe3a806bf00 msgr2=0x7fe3a806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.548+0000 7fe3ce800640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe3a806bf00 0x7fe3a806e3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4033000 tx=0x7fe3b4008030 comp rx=0 tx=0).stop 2024-04-24T22:39:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.548+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c806bc50 msgr2=0x7fe3c80b4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.548+0000 7fe3ce800640 1 --2- 172.21.15.47:0/2871598351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c806bc50 0x7fe3c80b4bc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800c3c0 tx=0x7fe3b8001e70 comp rx=0 tx=0).stop 2024-04-24T22:39:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.549+0000 7fe3c7e00640 1 -- 172.21.15.47:0/2871598351 reap_dead start 2024-04-24T22:39:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.549+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 shutdown_connections 2024-04-24T22:39:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.549+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 >> 172.21.15.47:0/2871598351 conn(0x7fe3c806c940 msgr2=0x7fe3c8072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.549+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 shutdown_connections 2024-04-24T22:39:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.549+0000 7fe3ce800640 1 -- 172.21.15.47:0/2871598351 wait complete. 2024-04-24T22:39:46.582 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-04-24T22:39:46.583 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T22:39:46.583 INFO:teuthology.orchestra.run.smithi047.stdout:55834574855 2024-04-24T22:39:46.584 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T22:39:46.584 INFO:teuthology.orchestra.run.smithi047.stdout:55834574855 2024-04-24T22:39:46.585 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T22:39:46.585 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-04-24T22:39:46.586 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T22:39:46.586 INFO:teuthology.orchestra.run.smithi047.stdout:55834574855 2024-04-24T22:39:46.587 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T22:39:46.587 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-04-24T22:39:46.588 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T22:39:46.588 INFO:teuthology.orchestra.run.smithi047.stdout:55834574855 2024-04-24T22:39:46.589 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T22:39:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.910+0000 7fa493600640 1 Processor -- start 2024-04-24T22:39:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.910+0000 7fa493600640 1 -- start start 2024-04-24T22:39:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.910+0000 7f6973e00640 1 Processor -- start 2024-04-24T22:39:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.910+0000 7f6973e00640 1 -- start start 2024-04-24T22:39:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.912+0000 7fa493600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c0cc840 0x7fa48c0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.912+0000 7fa493600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa48c073380 0x7fa48c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.912+0000 7fa493600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa48c15b240 con 0x7fa48c0cb6d0 2024-04-24T22:39:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.912+0000 7fa493600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa48c15daa0 con 0x7fa48c073380 2024-04-24T22:39:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.912+0000 7fa493600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c15dcf0 con 0x7fa48c0cc840 2024-04-24T22:39:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.913+0000 7fa490c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c0cc840 0x7fa48c0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.913+0000 7fa490c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c0cc840 0x7fa48c0ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:59108/0 (socket says 172.21.15.47:59108) 2024-04-24T22:39:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.913+0000 7f6973e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f696c0b4fa0 0x7f696c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.913+0000 7f6973e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f696c0c5740 0x7f696c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.913+0000 7fa490c00640 1 -- 172.21.15.47:0/2005911368 learned_addr learned my addr 172.21.15.47:0/2005911368 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.913+0000 7f6973e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f696c0b2d20 con 0x7f696c16b450 2024-04-24T22:39:46.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.913+0000 7f6973e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f696c0b2f90 con 0x7f696c0c5740 2024-04-24T22:39:46.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.913+0000 7f6973e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f696c0b3270 con 0x7f696c0b4fa0 2024-04-24T22:39:46.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.914+0000 7fa48be00640 1 --2- 172.21.15.47:0/2005911368 >> v2:172.21.15.47:3300/0 conn(0x7fa48c073380 0x7fa48c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.914+0000 7fa48be00640 1 -- 172.21.15.47:0/2005911368 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c0cc840 msgr2=0x7fa48c0ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.914+0000 7fa48be00640 1 --2- 172.21.15.47:0/2005911368 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c0cc840 0x7fa48c0ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.914+0000 7fa48be00640 1 -- 172.21.15.47:0/2005911368 >> v1:172.21.15.47:6789/0 conn(0x7fa48c0cb6d0 legacy=0x7fa48c0cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.914+0000 7f6971e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f696c0c5740 0x7f696c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.914+0000 7f6971e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f696c0c5740 0x7f696c06b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54004/0 (socket says 172.21.15.47:54004) 2024-04-24T22:39:46.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.914+0000 7f6971e00640 1 -- 172.21.15.47:0/376191336 learned_addr learned my addr 172.21.15.47:0/376191336 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.915+0000 7f6970a00640 1 --2- 172.21.15.47:0/376191336 >> v2:172.21.15.113:3300/0 conn(0x7f696c0b4fa0 0x7f696c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.916+0000 7f696be00640 1 -- 172.21.15.47:0/376191336 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277485218 0 0) 0x7f696c0b2d20 con 0x7f696c16b450 2024-04-24T22:39:46.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.916+0000 7f696be00640 1 -- 172.21.15.47:0/376191336 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6954003520 con 0x7f696c16b450 2024-04-24T22:39:46.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.916+0000 7f6970a00640 1 -- 172.21.15.47:0/376191336 >> v2:172.21.15.47:3300/0 conn(0x7f696c0c5740 msgr2=0x7f696c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.916+0000 7f6970a00640 1 --2- 172.21.15.47:0/376191336 >> v2:172.21.15.47:3300/0 conn(0x7f696c0c5740 0x7f696c06b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.916+0000 7f6970a00640 1 -- 172.21.15.47:0/376191336 >> v1:172.21.15.47:6789/0 conn(0x7f696c16b450 legacy=0x7f696c0c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.916+0000 7f6970a00640 1 -- 172.21.15.47:0/376191336 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c0b3550 con 0x7f696c0b4fa0 2024-04-24T22:39:46.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.915+0000 7fa48be00640 1 -- 172.21.15.47:0/2005911368 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c15dfd0 con 0x7fa48c073380 2024-04-24T22:39:46.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.915+0000 7fd867c00640 1 Processor -- start 2024-04-24T22:39:46.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.915+0000 7fd867c00640 1 -- start start 2024-04-24T22:39:46.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.917+0000 7f6970a00640 1 --2- 172.21.15.47:0/376191336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696c0b4fa0 0x7f696c06b440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f69580097a0 tx=0x7f695802a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b76d01d34d748afa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.918+0000 7fd865200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd860071de0 0x7fd8600c78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40896/0 (socket says 172.21.15.47:40896) 2024-04-24T22:39:46.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.918+0000 7fa48be00640 1 --2- 172.21.15.47:0/2005911368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa48c073380 0x7fa48c0ccc20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009a00 tx=0x7fa47c02a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6413e5c21c545efa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.918+0000 7fd867c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8600c6b00 0x7fd8601649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.918+0000 7fd865200640 1 -- 172.21.15.47:0/1781753285 learned_addr learned my addr 172.21.15.47:0/1781753285 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.918+0000 7f696be00640 1 -- 172.21.15.47:0/376191336 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958037070 con 0x7f696c0b4fa0 2024-04-24T22:39:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.918+0000 7f696be00640 1 -- 172.21.15.47:0/376191336 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6958032440 con 0x7f696c0b4fa0 2024-04-24T22:39:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.918+0000 7f696be00640 1 -- 172.21.15.47:0/376191336 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958032850 con 0x7f696c0b4fa0 2024-04-24T22:39:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7fd867c00640 1 --2- 172.21.15.47:0/1781753285 >> v2:172.21.15.47:3300/0 conn(0x7fd8601673d0 0x7fd860167830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7fd85fe00640 1 --2- 172.21.15.47:0/1781753285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8600c6b00 0x7fd8601649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7fd867c00640 1 -- 172.21.15.47:0/1781753285 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8600c6f40 con 0x7fd860071de0 2024-04-24T22:39:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7fd867c00640 1 -- 172.21.15.47:0/1781753285 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd8600c71b0 con 0x7fd8601673d0 2024-04-24T22:39:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7fd867c00640 1 -- 172.21.15.47:0/1781753285 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd8600c7490 con 0x7fd8600c6b00 2024-04-24T22:39:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7fa48b400640 1 -- 172.21.15.47:0/2005911368 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c037070 con 0x7fa48c073380 2024-04-24T22:39:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.920+0000 7fa48b400640 1 -- 172.21.15.47:0/2005911368 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c032440 con 0x7fa48c073380 2024-04-24T22:39:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.920+0000 7fa48b400640 1 -- 172.21.15.47:0/2005911368 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c032810 con 0x7fa48c073380 2024-04-24T22:39:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.920+0000 7fd85f400640 1 -- 172.21.15.47:0/1781753285 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 83270179 0 0) 0x7fd8600c6f40 con 0x7fd860071de0 2024-04-24T22:39:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.920+0000 7fd865c00640 1 --2- 172.21.15.47:0/1781753285 >> v2:172.21.15.47:3300/0 conn(0x7fd8601673d0 0x7fd860167830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.920+0000 7fd85fe00640 1 -- 172.21.15.47:0/1781753285 >> v2:172.21.15.47:3300/0 conn(0x7fd8601673d0 msgr2=0x7fd860167830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.920+0000 7fd85fe00640 1 --2- 172.21.15.47:0/1781753285 >> v2:172.21.15.47:3300/0 conn(0x7fd8601673d0 0x7fd860167830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.920+0000 7fd85fe00640 1 -- 172.21.15.47:0/1781753285 >> v1:172.21.15.47:6789/0 conn(0x7fd860071de0 legacy=0x7fd8600c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.920+0000 7fd85fe00640 1 -- 172.21.15.47:0/1781753285 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86016a9e0 con 0x7fd8600c6b00 2024-04-24T22:39:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7f6973e00640 1 -- 172.21.15.47:0/376191336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696c0b4fa0 msgr2=0x7f696c06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7f6973e00640 1 --2- 172.21.15.47:0/376191336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696c0b4fa0 0x7f696c06b440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f69580097a0 tx=0x7f695802a2a0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7f6973e00640 1 -- 172.21.15.47:0/376191336 shutdown_connections 2024-04-24T22:39:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7f6973e00640 1 --2- 172.21.15.47:0/376191336 >> v2:172.21.15.47:3300/0 conn(0x7f696c0c5740 0x7f696c06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7f6973e00640 1 --2- 172.21.15.47:0/376191336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696c0b4fa0 0x7f696c06b440 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7f6973e00640 1 -- 172.21.15.47:0/376191336 >> 172.21.15.47:0/376191336 conn(0x7f696c0d0270 msgr2=0x7f696c0d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.919+0000 7f6973e00640 1 -- 172.21.15.47:0/376191336 shutdown_connections 2024-04-24T22:39:46.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.920+0000 7f6973e00640 1 -- 172.21.15.47:0/376191336 wait complete. 2024-04-24T22:39:46.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.922+0000 7fd85fe00640 1 --2- 172.21.15.47:0/1781753285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8600c6b00 0x7fd8601649c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0097a0 tx=0x7fd84c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85269da53fd754d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.922+0000 7fa493600640 1 -- 172.21.15.47:0/2005911368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa48c073380 msgr2=0x7fa48c0ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.922+0000 7fa493600640 1 --2- 172.21.15.47:0/2005911368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa48c073380 0x7fa48c0ccc20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009a00 tx=0x7fa47c02a3b0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.923+0000 7f6973e00640 1 Processor -- start 2024-04-24T22:39:46.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.923+0000 7fd85f400640 1 -- 172.21.15.47:0/1781753285 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c037070 con 0x7fd8600c6b00 2024-04-24T22:39:46.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.923+0000 7f6973e00640 1 -- start start 2024-04-24T22:39:46.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.923+0000 7fd85f400640 1 -- 172.21.15.47:0/1781753285 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84c032440 con 0x7fd8600c6b00 2024-04-24T22:39:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.923+0000 7fd85f400640 1 -- 172.21.15.47:0/1781753285 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c032850 con 0x7fd8600c6b00 2024-04-24T22:39:46.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.923+0000 7fa493600640 1 -- 172.21.15.47:0/2005911368 shutdown_connections 2024-04-24T22:39:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.923+0000 7fa493600640 1 --2- 172.21.15.47:0/2005911368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa48c073380 0x7fa48c0ccc20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.923+0000 7fa493600640 1 --2- 172.21.15.47:0/2005911368 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c0cc840 0x7fa48c0ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.923+0000 7fa493600640 1 -- 172.21.15.47:0/2005911368 >> 172.21.15.47:0/2005911368 conn(0x7fa48c06c940 msgr2=0x7fa48c072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.923+0000 7fa493600640 1 -- 172.21.15.47:0/2005911368 shutdown_connections 2024-04-24T22:39:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.924+0000 7fd867c00640 1 -- 172.21.15.47:0/1781753285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8600c6b00 msgr2=0x7fd8601649c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.924+0000 7fd867c00640 1 --2- 172.21.15.47:0/1781753285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8600c6b00 0x7fd8601649c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0097a0 tx=0x7fd84c02a1a0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.924+0000 7fd867c00640 1 -- 172.21.15.47:0/1781753285 shutdown_connections 2024-04-24T22:39:46.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.924+0000 7fd867c00640 1 --2- 172.21.15.47:0/1781753285 >> v2:172.21.15.47:3300/0 conn(0x7fd8601673d0 0x7fd860167830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.924+0000 7fd867c00640 1 --2- 172.21.15.47:0/1781753285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8600c6b00 0x7fd8601649c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.924+0000 7fd867c00640 1 -- 172.21.15.47:0/1781753285 >> 172.21.15.47:0/1781753285 conn(0x7fd86006e580 msgr2=0x7fd86006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.924+0000 7fd867c00640 1 -- 172.21.15.47:0/1781753285 shutdown_connections 2024-04-24T22:39:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.925+0000 7f6973e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f696c0b4fa0 0x7f696c1fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7f6973e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f696c0c5740 0x7f696c0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7f6973e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696c16b450 0x7f696c0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7f6973e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f696c16b830 con 0x7f696c0b4fa0 2024-04-24T22:39:46.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7f6973e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f696c0c5c80 con 0x7f696c16b450 2024-04-24T22:39:46.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7f6973e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f696c0c7530 con 0x7f696c0c5740 2024-04-24T22:39:46.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7f6971400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f696c0b4fa0 0x7f696c1fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7f6971400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f696c0b4fa0 0x7f696c1fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54008/0 (socket says 172.21.15.47:54008) 2024-04-24T22:39:46.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7f6971400640 1 -- 172.21.15.47:0/2387203116 learned_addr learned my addr 172.21.15.47:0/2387203116 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7f6970a00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f696c0c5740 0x7f696c0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.925+0000 7fd867c00640 1 -- 172.21.15.47:0/1781753285 wait complete. 2024-04-24T22:39:46.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.924+0000 7fa493600640 1 -- 172.21.15.47:0/2005911368 wait complete. 2024-04-24T22:39:46.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7fa493600640 1 Processor -- start 2024-04-24T22:39:46.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.926+0000 7f6971e00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696c16b450 0x7f696c0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.927+0000 7f6970a00640 1 -- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696c16b450 msgr2=0x7f696c0c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.927+0000 7f6970a00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696c16b450 0x7f696c0c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.927+0000 7f6970a00640 1 -- 172.21.15.47:0/2387203116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f696c0b4fa0 msgr2=0x7f696c1fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.927+0000 7fa493600640 1 -- start start 2024-04-24T22:39:46.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.927+0000 7f6970a00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f696c0b4fa0 0x7f696c1fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.927+0000 7f6970a00640 1 -- 172.21.15.47:0/2387203116 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c1f3490 con 0x7f696c0c5740 2024-04-24T22:39:46.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.927+0000 7f6971400640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f696c0b4fa0 0x7f696c1fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:46.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.928+0000 7f6970a00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f696c0c5740 0x7f696c0cf1d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6958031000 tx=0x7f69580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.928+0000 7fd867c00640 1 Processor -- start 2024-04-24T22:39:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.928+0000 7f6969600640 1 -- 172.21.15.47:0/2387203116 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958037050 con 0x7f696c0c5740 2024-04-24T22:39:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.928+0000 7fd867c00640 1 -- start start 2024-04-24T22:39:46.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7f6969600640 1 -- 172.21.15.47:0/2387203116 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6958038040 con 0x7f696c0c5740 2024-04-24T22:39:46.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7f6973e00640 1 -- 172.21.15.47:0/2387203116 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696c200b90 con 0x7f696c0c5740 2024-04-24T22:39:46.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7f6969600640 1 -- 172.21.15.47:0/2387203116 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695803c8d0 con 0x7f696c0c5740 2024-04-24T22:39:46.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa493600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c073380 0x7fa48c0c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa493600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa48c0cb6d0 0x7fa48c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7f6973e00640 1 -- 172.21.15.47:0/2387203116 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f696c201150 con 0x7f696c0c5740 2024-04-24T22:39:46.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa493600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0cc840 0x7fa48c0c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa493600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c15e870 con 0x7fa48c0cb6d0 2024-04-24T22:39:46.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa493600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c15e590 con 0x7fa48c0cc840 2024-04-24T22:39:46.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa493600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c15af30 con 0x7fa48c073380 2024-04-24T22:39:46.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.930+0000 7ff635c00640 1 Processor -- start 2024-04-24T22:39:46.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa491600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0cc840 0x7fa48c0c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa490c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c073380 0x7fa48c0c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa491600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0cc840 0x7fa48c0c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37814/0 (socket says 172.21.15.47:37814) 2024-04-24T22:39:46.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa491600640 1 -- 172.21.15.47:0/2005417635 learned_addr learned my addr 172.21.15.47:0/2005417635 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.929+0000 7fa48be00640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa48c0cb6d0 0x7fa48c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.930+0000 7fa491600640 1 -- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c073380 msgr2=0x7fa48c0c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.930+0000 7fa491600640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c073380 0x7fa48c0c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.930+0000 7fa491600640 1 -- 172.21.15.47:0/2005417635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa48c0cb6d0 msgr2=0x7fa48c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.930+0000 7ff635c00640 1 -- start start 2024-04-24T22:39:46.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.930+0000 7fa491600640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa48c0cb6d0 0x7fa48c15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.930+0000 7fa491600640 1 -- 172.21.15.47:0/2005417635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c0c9c40 con 0x7fa48c0cc840 2024-04-24T22:39:46.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.931+0000 7fa48be00640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa48c0cb6d0 0x7fa48c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:46.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.931+0000 7fa491600640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0cc840 0x7fa48c0c9300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c7b0 tx=0x7fa48400cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.931+0000 7f6973e00640 1 -- 172.21.15.47:0/2387203116 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696c073b90 con 0x7f696c0c5740 2024-04-24T22:39:46.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fd867c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd860071de0 0x7fd86015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fd867c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8600c6b00 0x7fd86015b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fd867c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8601673d0 0x7fd86015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fd867c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd86016b280 con 0x7fd8600c6b00 2024-04-24T22:39:46.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fd867c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd86016afa0 con 0x7fd860071de0 2024-04-24T22:39:46.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fd867c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd8600c8000 con 0x7fd8601673d0 2024-04-24T22:39:46.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fc0bce00640 1 Processor -- start 2024-04-24T22:39:46.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fc0bce00640 1 -- start start 2024-04-24T22:39:46.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fa488c00640 1 -- 172.21.15.47:0/2005417635 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484004b60 con 0x7fa48c0cc840 2024-04-24T22:39:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fd865200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd860071de0 0x7fd86015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fa488c00640 1 -- 172.21.15.47:0/2005417635 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa484004d00 con 0x7fa48c0cc840 2024-04-24T22:39:46.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fa488c00640 1 -- 172.21.15.47:0/2005417635 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48400aa90 con 0x7fa48c0cc840 2024-04-24T22:39:46.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fd865200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd860071de0 0x7fd86015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37820/0 (socket says 172.21.15.47:37820) 2024-04-24T22:39:46.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fd865200640 1 -- 172.21.15.47:0/3163830156 learned_addr learned my addr 172.21.15.47:0/3163830156 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fa493600640 1 -- 172.21.15.47:0/2005417635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c0c9f20 con 0x7fa48c0cc840 2024-04-24T22:39:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7ff635c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff630158fc0 0x7ff6300c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7ff635c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 0x7ff6300c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7ff635c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6301621d0 con 0x7ff630158be0 2024-04-24T22:39:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7ff635c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff6301624b0 con 0x7ff630158fc0 2024-04-24T22:39:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7ff635c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff630162790 con 0x7ff6300c1280 2024-04-24T22:39:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.932+0000 7fd865c00640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8601673d0 0x7fd86015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.933+0000 7fa493600640 1 -- 172.21.15.47:0/2005417635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c1fda30 con 0x7fa48c0cc840 2024-04-24T22:39:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.933+0000 7ff62e000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff630158fc0 0x7ff6300c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fd85fe00640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8600c6b00 0x7fd86015b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fc0bce00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8154560 0x7fc0b80cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fc0bce00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0b8073280 0x7fc0b80cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fd85fe00640 1 -- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8601673d0 msgr2=0x7fd86015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7ff62ea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff630158be0 0x7ff6300c78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:51778/0 (socket says 172.21.15.47:51778) 2024-04-24T22:39:46.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fc0bce00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0b815b2c0 con 0x7fc0b80cc660 2024-04-24T22:39:46.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fc0bce00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b815da90 con 0x7fc0b8073280 2024-04-24T22:39:46.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fc0bce00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b815dd70 con 0x7fc0b8154560 2024-04-24T22:39:46.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fc0b7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc0b80cc660 0x7fc0b80ccaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:51794/0 (socket says 172.21.15.47:51794) 2024-04-24T22:39:46.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.935+0000 7fc0b7e00640 1 -- 172.21.15.47:0/2557063396 learned_addr learned my addr 172.21.15.47:0/2557063396 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:46.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7ff62ea00640 1 -- 172.21.15.47:0/1417053447 learned_addr learned my addr 172.21.15.47:0/1417053447 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:46.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7ff62f400640 1 --2- 172.21.15.47:0/1417053447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 0x7ff6300c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7ff62e000640 1 -- 172.21.15.47:0/1417053447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 msgr2=0x7ff6300c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.935+0000 7fd35a000640 1 Processor -- start 2024-04-24T22:39:46.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fd35a000640 1 -- start start 2024-04-24T22:39:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.935+0000 7fc0b7400640 1 --2- 172.21.15.47:0/2557063396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8154560 0x7fc0b80cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.935+0000 7fc0b7400640 1 -- 172.21.15.47:0/2557063396 >> v2:172.21.15.47:3300/0 conn(0x7fc0b8073280 msgr2=0x7fc0b80cb970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:39:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.935+0000 7fc0b6000640 1 -- 172.21.15.47:0/2557063396 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226604789 0 0) 0x7fc0b815b2c0 con 0x7fc0b80cc660 2024-04-24T22:39:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fc0b6a00640 1 --2- 172.21.15.47:0/2557063396 >> v2:172.21.15.47:3300/0 conn(0x7fc0b8073280 0x7fc0b80cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fc0b7400640 1 --2- 172.21.15.47:0/2557063396 >> v2:172.21.15.47:3300/0 conn(0x7fc0b8073280 0x7fc0b80cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fc0b7400640 1 -- 172.21.15.47:0/2557063396 >> v1:172.21.15.113:6789/0 conn(0x7fc0b80cc660 legacy=0x7fc0b80ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fc0b7400640 1 -- 172.21.15.47:0/2557063396 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b815e050 con 0x7fc0b8154560 2024-04-24T22:39:46.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.935+0000 7f6969600640 1 -- 172.21.15.47:0/2387203116 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6958045020 con 0x7f696c0c5740 2024-04-24T22:39:46.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fc0b6a00640 1 --2- 172.21.15.47:0/2557063396 >> v2:172.21.15.47:3300/0 conn(0x7fc0b8073280 0x7fc0b80cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:39:46.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7ff62e000640 1 --2- 172.21.15.47:0/1417053447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 0x7ff6300c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7ff62e000640 1 -- 172.21.15.47:0/1417053447 >> v1:172.21.15.113:6789/0 conn(0x7ff630158be0 legacy=0x7ff6300c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7ff62e000640 1 -- 172.21.15.47:0/1417053447 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff630162a70 con 0x7ff630158fc0 2024-04-24T22:39:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.937+0000 7fa488c00640 1 -- 172.21.15.47:0/2005417635 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa48400ac30 con 0x7fa48c0cc840 2024-04-24T22:39:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.937+0000 7fc0b7400640 1 --2- 172.21.15.47:0/2557063396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8154560 0x7fc0b80cb430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00097a0 tx=0x7fc0a002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4286d4da39e65fda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.937+0000 7ff62e000640 1 --2- 172.21.15.47:0/1417053447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff630158fc0 0x7ff6300c8590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0097a0 tx=0x7ff61c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca00206874f590 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fd85fe00640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8601673d0 0x7fd86015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fd85fe00640 1 -- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd860071de0 msgr2=0x7fd86015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.934+0000 7fd85fe00640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd860071de0 0x7fd86015dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.935+0000 7fd85fe00640 1 -- 172.21.15.47:0/3163830156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8601fdbe0 con 0x7fd8600c6b00 2024-04-24T22:39:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.935+0000 7fd85fe00640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8600c6b00 0x7fd86015b9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd84c031000 tx=0x7fd84c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fd85cc00640 1 -- 172.21.15.47:0/3163830156 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c037050 con 0x7fd8600c6b00 2024-04-24T22:39:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fd85cc00640 1 -- 172.21.15.47:0/3163830156 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84c038040 con 0x7fd8600c6b00 2024-04-24T22:39:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fd85cc00640 1 -- 172.21.15.47:0/3163830156 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c030aa0 con 0x7fd8600c6b00 2024-04-24T22:39:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fd867c00640 1 -- 172.21.15.47:0/3163830156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86015c6f0 con 0x7fd8600c6b00 2024-04-24T22:39:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.936+0000 7fd867c00640 1 -- 172.21.15.47:0/3163830156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8600c1d90 con 0x7fd8600c6b00 2024-04-24T22:39:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7fd35a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd354154560 0x7fd3540cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7fd35a000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd3540cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7fa493600640 1 -- 172.21.15.47:0/2005417635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48c073b90 con 0x7fa48c0cc840 2024-04-24T22:39:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7fd353e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd3540cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7fd353e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd3540cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:59168/0 (socket says 172.21.15.47:59168) 2024-04-24T22:39:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7fd358c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd354154560 0x7fd3540cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7fd358c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd354154560 0x7fd3540cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37828/0 (socket says 172.21.15.47:37828) 2024-04-24T22:39:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7fd35a000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35415b2c0 con 0x7fd3540cc660 2024-04-24T22:39:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7fd35a000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd35415da90 con 0x7fd354154560 2024-04-24T22:39:46.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7fd35a000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd35415dd70 con 0x7fd354073280 2024-04-24T22:39:46.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.939+0000 7fd353e00640 1 -- 172.21.15.47:0/4219390581 learned_addr learned my addr 172.21.15.47:0/4219390581 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.938+0000 7f6969600640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f694c06bf00 0x7f694c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.939+0000 7fd852000640 1 -- 172.21.15.47:0/3163830156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd834005290 con 0x7fd8600c6b00 2024-04-24T22:39:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.939+0000 7f6969600640 1 -- 172.21.15.47:0/2387203116 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f69580ab1c0 con 0x7f696c0c5740 2024-04-24T22:39:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.939+0000 7fd353e00640 1 -- 172.21.15.47:0/4219390581 >> v2:172.21.15.113:3300/0 conn(0x7fd354154560 msgr2=0x7fd3540cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.939+0000 7fd353e00640 1 --2- 172.21.15.47:0/4219390581 >> v2:172.21.15.113:3300/0 conn(0x7fd354154560 0x7fd3540cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.939+0000 7fd353e00640 1 -- 172.21.15.47:0/4219390581 >> v1:172.21.15.47:6789/0 conn(0x7fd3540cc660 legacy=0x7fd3540ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.939+0000 7ff62d600640 1 -- 172.21.15.47:0/1417053447 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c037070 con 0x7ff630158fc0 2024-04-24T22:39:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.939+0000 7ff62d600640 1 -- 172.21.15.47:0/1417053447 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61c032440 con 0x7ff630158fc0 2024-04-24T22:39:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.939+0000 7ff62d600640 1 -- 172.21.15.47:0/1417053447 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c02ae60 con 0x7ff630158fc0 2024-04-24T22:39:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.939+0000 7f6971400640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f694c06bf00 0x7f694c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.940+0000 7f6971400640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f694c06bf00 0x7f694c06e3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f696000f290 tx=0x7f6960009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.940+0000 7fc0b6000640 1 -- 172.21.15.47:0/2557063396 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0037070 con 0x7fc0b8154560 2024-04-24T22:39:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.940+0000 7fc0b6000640 1 -- 172.21.15.47:0/2557063396 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a0032440 con 0x7fc0b8154560 2024-04-24T22:39:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.940+0000 7fc0b6000640 1 -- 172.21.15.47:0/2557063396 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0032810 con 0x7fc0b8154560 2024-04-24T22:39:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.941+0000 7f6969600640 1 -- 172.21.15.47:0/2387203116 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6958079be0 con 0x7f696c0c5740 2024-04-24T22:39:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.940+0000 7fd353e00640 1 -- 172.21.15.47:0/4219390581 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35415e050 con 0x7fd354073280 2024-04-24T22:39:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.941+0000 7fc0bce00640 1 -- 172.21.15.47:0/2557063396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8154560 msgr2=0x7fc0b80cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.941+0000 7fc0bce00640 1 --2- 172.21.15.47:0/2557063396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8154560 0x7fc0b80cb430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00097a0 tx=0x7fc0a002a1a0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.942+0000 7fc0bce00640 1 -- 172.21.15.47:0/2557063396 shutdown_connections 2024-04-24T22:39:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.942+0000 7fc0bce00640 1 --2- 172.21.15.47:0/2557063396 >> v2:172.21.15.47:3300/0 conn(0x7fc0b8073280 0x7fc0b80cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.942+0000 7fc0bce00640 1 --2- 172.21.15.47:0/2557063396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8154560 0x7fc0b80cb430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.942+0000 7fc0bce00640 1 -- 172.21.15.47:0/2557063396 >> 172.21.15.47:0/2557063396 conn(0x7fc0b80672c0 msgr2=0x7fc0b8072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.941+0000 7fa488c00640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa46c06be80 0x7fa46c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.941+0000 7fa488c00640 1 -- 172.21.15.47:0/2005417635 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa48408b340 con 0x7fa48c0cc840 2024-04-24T22:39:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.941+0000 7fd85cc00640 1 -- 172.21.15.47:0/3163830156 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd84c045020 con 0x7fd8600c6b00 2024-04-24T22:39:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.942+0000 7fd353e00640 1 --2- 172.21.15.47:0/4219390581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd3540cb970 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009870 tx=0x7fd34c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edacfae8386904fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.942+0000 7fa490c00640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa46c06be80 0x7fa46c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7fd85cc00640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd84006be30 0x7fd84006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7fd85cc00640 1 -- 172.21.15.47:0/3163830156 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd84c0ab110 con 0x7fd8600c6b00 2024-04-24T22:39:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7ff635c00640 1 -- 172.21.15.47:0/1417053447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff630158fc0 msgr2=0x7ff6300c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7ff635c00640 1 --2- 172.21.15.47:0/1417053447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff630158fc0 0x7ff6300c8590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0097a0 tx=0x7ff61c02a150 comp rx=0 tx=0).stop 2024-04-24T22:39:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7fd353400640 1 -- 172.21.15.47:0/4219390581 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c037070 con 0x7fd354073280 2024-04-24T22:39:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7fd353400640 1 -- 172.21.15.47:0/4219390581 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c032440 con 0x7fd354073280 2024-04-24T22:39:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7fd353400640 1 -- 172.21.15.47:0/4219390581 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c032810 con 0x7fd354073280 2024-04-24T22:39:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7fd865200640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd84006be30 0x7fd84006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7ff635c00640 1 -- 172.21.15.47:0/1417053447 shutdown_connections 2024-04-24T22:39:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7ff635c00640 1 --2- 172.21.15.47:0/1417053447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 0x7ff6300c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7ff635c00640 1 --2- 172.21.15.47:0/1417053447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff630158fc0 0x7ff6300c8590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.943+0000 7ff635c00640 1 -- 172.21.15.47:0/1417053447 >> 172.21.15.47:0/1417053447 conn(0x7ff63006e580 msgr2=0x7ff63006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.942+0000 7fc0bce00640 1 -- 172.21.15.47:0/2557063396 shutdown_connections 2024-04-24T22:39:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.942+0000 7fc0bce00640 1 -- 172.21.15.47:0/2557063396 wait complete. 2024-04-24T22:39:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.944+0000 7ff635c00640 1 -- 172.21.15.47:0/1417053447 shutdown_connections 2024-04-24T22:39:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.944+0000 7ff635c00640 1 -- 172.21.15.47:0/1417053447 wait complete. 2024-04-24T22:39:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.945+0000 7fa488c00640 1 -- 172.21.15.47:0/2005417635 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa484059d60 con 0x7fa48c0cc840 2024-04-24T22:39:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.945+0000 7fe329a00640 1 Processor -- start 2024-04-24T22:39:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.945+0000 7fc0bce00640 1 Processor -- start 2024-04-24T22:39:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.945+0000 7fd35a000640 1 -- 172.21.15.47:0/4219390581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 msgr2=0x7fd3540cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.945+0000 7fd35a000640 1 --2- 172.21.15.47:0/4219390581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd3540cb970 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009870 tx=0x7fd34c02a270 comp rx=0 tx=0).stop 2024-04-24T22:39:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.946+0000 7fd865200640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd84006be30 0x7fd84006e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd854005f60 tx=0x7fd854000f10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.946+0000 7fe329a00640 1 -- start start 2024-04-24T22:39:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.946+0000 7ff635c00640 1 Processor -- start 2024-04-24T22:39:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7ff635c00640 1 -- start start 2024-04-24T22:39:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fe329a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe324152960 0x7fe3240c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.945+0000 7fc0bce00640 1 -- start start 2024-04-24T22:39:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fc0bce00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8073280 0x7fc0b81439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fc0bce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0b80cc660 0x7fc0b81495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fc0bce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0b8154560 0x7fc0b81522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fc0bce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b80cd200 con 0x7fc0b8154560 2024-04-24T22:39:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fc0bce00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b80caad0 con 0x7fc0b80cc660 2024-04-24T22:39:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fc0bce00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b80cd380 con 0x7fc0b8073280 2024-04-24T22:39:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fc0b6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0b80cc660 0x7fc0b81495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fc0b6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0b80cc660 0x7fc0b81495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37836/0 (socket says 172.21.15.47:37836) 2024-04-24T22:39:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fc0b6a00640 1 -- 172.21.15.47:0/1972396272 learned_addr learned my addr 172.21.15.47:0/1972396272 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fe329a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3240c9400 con 0x7fe3240c8fa0 2024-04-24T22:39:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fe329a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3240c7e50 con 0x7fe324072b80 2024-04-24T22:39:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.947+0000 7fe329a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe324158f20 con 0x7fe324152960 2024-04-24T22:39:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.948+0000 7fc0b7400640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8073280 0x7fc0b81439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.948+0000 7fc0b6a00640 1 -- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8073280 msgr2=0x7fc0b81439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.948+0000 7fa490c00640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa46c06be80 0x7fa46c06e340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa47800c900 tx=0x7fa478008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.948+0000 7fe322000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe3240c8fa0 0x7fe32415ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40914/0 (socket says 172.21.15.47:40914) 2024-04-24T22:39:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.948+0000 7fe322000640 1 -- 172.21.15.47:0/2418462958 learned_addr learned my addr 172.21.15.47:0/2418462958 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.948+0000 7fe322a00640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe324152960 0x7fe3240c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.948+0000 7fc0b7e00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0b8154560 0x7fc0b81522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.948+0000 7fc0b6a00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8073280 0x7fc0b81439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.948+0000 7fc0b6a00640 1 -- 172.21.15.47:0/1972396272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0b8154560 msgr2=0x7fc0b81522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.948+0000 7fc0b6a00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0b8154560 0x7fc0b81522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fc0b6a00640 1 -- 172.21.15.47:0/1972396272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b815d7a0 con 0x7fc0b80cc660 2024-04-24T22:39:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fc0b7400640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8073280 0x7fc0b81439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:39:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fc0b7e00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0b8154560 0x7fc0b81522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:39:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1742954429 0 0) 0x7fe3240c7e50 con 0x7fe324072b80 2024-04-24T22:39:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe308003520 con 0x7fe324072b80 2024-04-24T22:39:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 709374746 0 0) 0x7fe308003520 con 0x7fe324072b80 2024-04-24T22:39:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe324152960 msgr2=0x7fe3240c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe324152960 0x7fe3240c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 >> v1:172.21.15.47:6789/0 conn(0x7fe3240c8fa0 legacy=0x7fe32415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3241585f0 con 0x7fe324072b80 2024-04-24T22:39:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 567141988 0 0) 0x7fe314001670 con 0x7fe324072b80 2024-04-24T22:39:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 >> v1:172.21.15.113:6789/0 conn(0x7fe324072b80 legacy=0x7fe324162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3080039b0 0x7fe308003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308004800 0x7fe308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3080051c0 0x7fe30800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe324158f20 con 0x7fe3080051c0 2024-04-24T22:39:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe30800bf90 con 0x7fe3080039b0 2024-04-24T22:39:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe30800c190 con 0x7fe308004800 2024-04-24T22:39:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7fe323400640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3080051c0 0x7fe30800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7fe322a00640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3080039b0 0x7fe308003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.949+0000 7ff635c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 0x7ff630161690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7ff635c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff630158be0 0x7ff63015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7ff635c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff630158fc0 0x7ff63015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7ff635c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff63015ed50 con 0x7ff630158fc0 2024-04-24T22:39:46.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7ff635c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff63015ebd0 con 0x7ff630158be0 2024-04-24T22:39:46.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7ff635c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff63015eed0 con 0x7ff6300c1280 2024-04-24T22:39:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7fe323400640 1 -- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308004800 msgr2=0x7fe308004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T22:39:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7fe323400640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308004800 0x7fe308004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7fe323400640 1 -- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3080039b0 msgr2=0x7fe308003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7fe323400640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3080039b0 0x7fe308003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7fe323400640 1 -- 172.21.15.47:0/2418462958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3241585f0 con 0x7fe3080051c0 2024-04-24T22:39:46.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7ff62ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 0x7ff630161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7ff62ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 0x7ff630161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:59202/0 (socket says 172.21.15.47:59202) 2024-04-24T22:39:46.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7ff62ea00640 1 -- 172.21.15.47:0/115830144 learned_addr learned my addr 172.21.15.47:0/115830144 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.950+0000 7ff62e000640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff630158be0 0x7ff63015aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7fe322a00640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3080039b0 0x7fe308003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:46.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7fd35a000640 1 -- 172.21.15.47:0/4219390581 shutdown_connections 2024-04-24T22:39:46.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7fd35a000640 1 --2- 172.21.15.47:0/4219390581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd3540cb970 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7fd35a000640 1 --2- 172.21.15.47:0/4219390581 >> v2:172.21.15.113:3300/0 conn(0x7fd354154560 0x7fd3540cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7fd35a000640 1 -- 172.21.15.47:0/4219390581 >> 172.21.15.47:0/4219390581 conn(0x7fd3540672c0 msgr2=0x7fd354072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7fc0b6a00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0b80cc660 0x7fc0b81495b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800c7f0 tx=0x7fc0a800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7ff62f400640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff630158fc0 0x7ff63015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7fd35a000640 1 -- 172.21.15.47:0/4219390581 shutdown_connections 2024-04-24T22:39:46.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7fe323400640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3080051c0 0x7fe30800b9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c160 tx=0x7fe308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331f3343be415565 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7ff62e000640 1 -- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 msgr2=0x7ff630161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7ff62e000640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 0x7ff630161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7ff62e000640 1 -- 172.21.15.47:0/115830144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff630158fc0 msgr2=0x7ff63015aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7ff62e000640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff630158fc0 0x7ff63015aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:46.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7ff62e000640 1 -- 172.21.15.47:0/115830144 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6301f5060 con 0x7ff630158be0 2024-04-24T22:39:46.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.952+0000 7ff62e000640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff630158be0 0x7ff63015aab0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff61c031000 tx=0x7ff61c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:46.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.952+0000 7fd85cc00640 1 -- 172.21.15.47:0/3163830156 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd84c04c020 con 0x7fd8600c6b00 2024-04-24T22:39:46.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.953+0000 7ff626a00640 1 -- 172.21.15.47:0/115830144 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c037040 con 0x7ff630158be0 2024-04-24T22:39:46.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.953+0000 7ff626a00640 1 -- 172.21.15.47:0/115830144 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61c042070 con 0x7ff630158be0 2024-04-24T22:39:46.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.953+0000 7ff626a00640 1 -- 172.21.15.47:0/115830144 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c032830 con 0x7ff630158be0 2024-04-24T22:39:46.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.953+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314007c20 con 0x7fe3080051c0 2024-04-24T22:39:46.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.953+0000 7ff635c00640 1 -- 172.21.15.47:0/115830144 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6301f5300 con 0x7ff630158be0 2024-04-24T22:39:46.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.953+0000 7ff635c00640 1 -- 172.21.15.47:0/115830144 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff630159f60 con 0x7ff630158be0 2024-04-24T22:39:46.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.951+0000 7fd35a000640 1 -- 172.21.15.47:0/4219390581 wait complete. 2024-04-24T22:39:46.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.953+0000 7fe321600640 1 -- 172.21.15.47:0/2418462958 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe314031600 con 0x7fe3080051c0 2024-04-24T22:39:46.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.954+0000 7fe329a00640 1 -- 172.21.15.47:0/2418462958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3080051c0 msgr2=0x7fe30800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:46.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.954+0000 7fe329a00640 1 --2- 172.21.15.47:0/2418462958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3080051c0 0x7fe30800b9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c160 tx=0x7fe308005830 comp rx=0 tx=0).stop 2024-04-24T22:39:46.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.954+0000 7fe323400640 1 -- 172.21.15.47:0/2418462958 reap_dead start 2024-04-24T22:39:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.954+0000 7fd35a000640 1 Processor -- start 2024-04-24T22:39:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.954+0000 7fd35a000640 1 -- start start 2024-04-24T22:39:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.952+0000 7fc0af400640 1 -- 172.21.15.47:0/1972396272 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8015070 con 0x7fc0b80cc660 2024-04-24T22:39:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.952+0000 7fc0af400640 1 -- 172.21.15.47:0/1972396272 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a8007820 con 0x7fc0b80cc660 2024-04-24T22:39:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.952+0000 7fc0af400640 1 -- 172.21.15.47:0/1972396272 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8007bd0 con 0x7fc0b80cc660 2024-04-24T22:39:46.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.952+0000 7fc0bce00640 1 -- 172.21.15.47:0/1972396272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b8149af0 con 0x7fc0b80cc660 2024-04-24T22:39:46.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.952+0000 7fc0bce00640 1 -- 172.21.15.47:0/1972396272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b81f5020 con 0x7fc0b80cc660 2024-04-24T22:39:46.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.955+0000 7fc48f200640 1 Processor -- start 2024-04-24T22:39:46.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.954+0000 7fe329a00640 1 -- 172.21.15.47:0/2418462958 shutdown_connections 2024-04-24T22:39:46.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.955+0000 7fe329a00640 1 -- 172.21.15.47:0/2418462958 >> 172.21.15.47:0/2418462958 conn(0x7fe32406e580 msgr2=0x7fe32406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:46.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.955+0000 7ff635c00640 1 -- 172.21.15.47:0/115830144 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff630073b90 con 0x7ff630158be0 2024-04-24T22:39:46.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.955+0000 7fe329a00640 1 -- 172.21.15.47:0/2418462958 shutdown_connections 2024-04-24T22:39:46.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.955+0000 7fe329a00640 1 -- 172.21.15.47:0/2418462958 wait complete. 2024-04-24T22:39:46.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.956+0000 7ff626a00640 1 -- 172.21.15.47:0/115830144 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff61c0329d0 con 0x7ff630158be0 2024-04-24T22:39:46.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.956+0000 7fd35a000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd354151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.956+0000 7fd35a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3540cc660 0x7fd3541fa670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.956+0000 7fd35a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd354154560 0x7fd354200e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.956+0000 7fd35a000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd3540cd200 con 0x7fd354154560 2024-04-24T22:39:46.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.956+0000 7fd35a000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd3540caad0 con 0x7fd3540cc660 2024-04-24T22:39:46.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.956+0000 7fd35a000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd3540cd380 con 0x7fd354073280 2024-04-24T22:39:46.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.956+0000 7fd353e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3540cc660 0x7fd3541fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.956+0000 7fd353e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3540cc660 0x7fd3541fa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37854/0 (socket says 172.21.15.47:37854) 2024-04-24T22:39:46.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.956+0000 7fd353e00640 1 -- 172.21.15.47:0/2483301724 learned_addr learned my addr 172.21.15.47:0/2483301724 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:46.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.957+0000 7ff626a00640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff61006be60 0x7ff61006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.955+0000 7fc48f200640 1 -- start start 2024-04-24T22:39:46.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.957+0000 7fc48f200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc48804beb0 0x7fc48804c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.957+0000 7fc48f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc488046520 0x7fc488046970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:46.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.957+0000 7fc48f200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4880dd8c0 con 0x7fc4880dd4e0 2024-04-24T22:39:46.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.957+0000 7fc48f200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc4880e4060 con 0x7fc48804beb0 2024-04-24T22:39:46.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.957+0000 7fc48f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc488046eb0 con 0x7fc488046520 2024-04-24T22:39:46.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.957+0000 7ff626a00640 1 -- 172.21.15.47:0/115830144 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff61c0ab670 con 0x7ff630158be0 2024-04-24T22:39:46.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.958+0000 7ff62ea00640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff61006be60 0x7ff61006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.958+0000 7fc0af400640 1 -- 172.21.15.47:0/1972396272 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0a8018070 con 0x7fc0b80cc660 2024-04-24T22:39:46.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.958+0000 7fd358c00640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd354151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.958+0000 7fc48d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc48804beb0 0x7fc48804c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.958+0000 7fc48de00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc4880dd4e0 0x7fc488045570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40926/0 (socket says 172.21.15.47:40926) 2024-04-24T22:39:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.958+0000 7fc48de00640 1 -- 172.21.15.47:0/1105146468 learned_addr learned my addr 172.21.15.47:0/1105146468 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.959+0000 7fc48e800640 1 --2- 172.21.15.47:0/1105146468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc488046520 0x7fc488046970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:46.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.959+0000 7fc0bce00640 1 -- 172.21.15.47:0/1972396272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b80c6080 con 0x7fc0b80cc660 2024-04-24T22:39:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.959+0000 7fd358c00640 1 -- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3540cc660 msgr2=0x7fd3541fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.959+0000 7fd358c00640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3540cc660 0x7fd3541fa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.959+0000 7fd358c00640 1 -- 172.21.15.47:0/2483301724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd354154560 msgr2=0x7fd354200e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.959+0000 7fd358c00640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd354154560 0x7fd354200e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.959+0000 7fd358c00640 1 -- 172.21.15.47:0/2483301724 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3541fabb0 con 0x7fd354073280 2024-04-24T22:39:47.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.959+0000 7ff62ea00640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff61006be60 0x7ff61006e320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff620007d50 tx=0x7ff620007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.959+0000 7fe329a00640 1 Processor -- start 2024-04-24T22:39:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.959+0000 7fe329a00640 1 -- start start 2024-04-24T22:39:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fd358c00640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd354151130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00e870 tx=0x7fd33c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fc0af400640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc09406be80 0x7fc09406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fc48ca00640 1 -- 172.21.15.47:0/1105146468 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3981237183 0 0) 0x7fc480001110 con 0x7fc4880dd4e0 2024-04-24T22:39:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fc48ca00640 1 -- 172.21.15.47:0/1105146468 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc474003520 con 0x7fc4880dd4e0 2024-04-24T22:39:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fc0af400640 1 -- 172.21.15.47:0/1972396272 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc0a808ad70 con 0x7fc0b80cc660 2024-04-24T22:39:47.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fc48d400640 1 -- 172.21.15.47:0/1105146468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc488046520 msgr2=0x7fc488046970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:47.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fc48d400640 1 --2- 172.21.15.47:0/1105146468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc488046520 0x7fc488046970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fc48d400640 1 -- 172.21.15.47:0/1105146468 >> v1:172.21.15.47:6789/0 conn(0x7fc4880dd4e0 legacy=0x7fc488045570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fc48d400640 1 -- 172.21.15.47:0/1105146468 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4880e34e0 con 0x7fc48804beb0 2024-04-24T22:39:47.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fc48e800640 1 --2- 172.21.15.47:0/1105146468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc488046520 0x7fc488046970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:47.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fd350c00640 1 -- 172.21.15.47:0/2483301724 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c017070 con 0x7fd354073280 2024-04-24T22:39:47.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.960+0000 7fc0b7400640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc09406be80 0x7fc09406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fd35a000640 1 -- 172.21.15.47:0/2483301724 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3541517c0 con 0x7fd354073280 2024-04-24T22:39:47.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fd35a000640 1 -- 172.21.15.47:0/2483301724 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3540cded0 con 0x7fd354073280 2024-04-24T22:39:47.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fe329a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3080051c0 0x7fe324071d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fe329a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3241fda50 0x7fe324070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fe329a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe324153420 0x7fe3240c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fe329a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe3240c8130 con 0x7fe324153420 2024-04-24T22:39:47.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fe329a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241593b0 con 0x7fe3241fda50 2024-04-24T22:39:47.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fe329a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe324158bb0 con 0x7fe3080051c0 2024-04-24T22:39:47.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fd350c00640 1 -- 172.21.15.47:0/2483301724 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd33c00ee60 con 0x7fd354073280 2024-04-24T22:39:47.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fd350c00640 1 -- 172.21.15.47:0/2483301724 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c017070 con 0x7fd354073280 2024-04-24T22:39:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fe322000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3241fda50 0x7fe324070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fe322000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3241fda50 0x7fe324070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37856/0 (socket says 172.21.15.47:37856) 2024-04-24T22:39:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.961+0000 7fe322000640 1 -- 172.21.15.47:0/4135655109 learned_addr learned my addr 172.21.15.47:0/4135655109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7fe322a00640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3080051c0 0x7fe324071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7fe322000640 1 -- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3080051c0 msgr2=0x7fe324071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7fe322000640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3080051c0 0x7fe324071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7fe322000640 1 -- 172.21.15.47:0/4135655109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe324153420 msgr2=0x7fe3240c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7fc0b7400640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc09406be80 0x7fc09406e340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0034000 tx=0x7fc0a002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7fc48d400640 1 --2- 172.21.15.47:0/1105146468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc48804beb0 0x7fc48804c2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009a00 tx=0x7fc47c02a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6f9f2982ba1184 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7ff626a00640 1 -- 172.21.15.47:0/115830144 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff61c07a130 con 0x7ff630158be0 2024-04-24T22:39:47.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7fe323400640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe324153420 0x7fe3240c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7fe322000640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe324153420 0x7fe3240c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7fe322000640 1 -- 172.21.15.47:0/4135655109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe324070dd0 con 0x7fe3241fda50 2024-04-24T22:39:47.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.962+0000 7fe323400640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe324153420 0x7fe3240c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:39:47.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.963+0000 7fc48ca00640 1 -- 172.21.15.47:0/1105146468 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c036070 con 0x7fc48804beb0 2024-04-24T22:39:47.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.963+0000 7fc48ca00640 1 -- 172.21.15.47:0/1105146468 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47c031600 con 0x7fc48804beb0 2024-04-24T22:39:47.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.963+0000 7fe322000640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3241fda50 0x7fe324070890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe31000c480 tx=0x7fe31000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.963+0000 7fc48ca00640 1 -- 172.21.15.47:0/1105146468 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c036070 con 0x7fc48804beb0 2024-04-24T22:39:47.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.964+0000 7fd35a000640 1 -- 172.21.15.47:0/2483301724 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c005290 con 0x7fd354073280 2024-04-24T22:39:47.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.964+0000 7fc48f200640 1 -- 172.21.15.47:0/1105146468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc48804beb0 msgr2=0x7fc48804c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.964+0000 7fc48f200640 1 --2- 172.21.15.47:0/1105146468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc48804beb0 0x7fc48804c2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009a00 tx=0x7fc47c02a4a0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.964+0000 7fc48f200640 1 -- 172.21.15.47:0/1105146468 shutdown_connections 2024-04-24T22:39:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.964+0000 7fc48f200640 1 --2- 172.21.15.47:0/1105146468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc488046520 0x7fc488046970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.964+0000 7fc48f200640 1 --2- 172.21.15.47:0/1105146468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc48804beb0 0x7fc48804c2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.964+0000 7fc48f200640 1 -- 172.21.15.47:0/1105146468 >> 172.21.15.47:0/1105146468 conn(0x7fc4880505d0 msgr2=0x7fc4880509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.965+0000 7fe31aa00640 1 -- 172.21.15.47:0/4135655109 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310017070 con 0x7fe3241fda50 2024-04-24T22:39:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.965+0000 7fe31aa00640 1 -- 172.21.15.47:0/4135655109 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe310005ce0 con 0x7fe3241fda50 2024-04-24T22:39:47.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.965+0000 7fe31aa00640 1 -- 172.21.15.47:0/4135655109 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3100072d0 con 0x7fe3241fda50 2024-04-24T22:39:47.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.964+0000 7fc48f200640 1 -- 172.21.15.47:0/1105146468 shutdown_connections 2024-04-24T22:39:47.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.964+0000 7fc48f200640 1 -- 172.21.15.47:0/1105146468 wait complete. 2024-04-24T22:39:47.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.965+0000 7fe329a00640 1 -- 172.21.15.47:0/4135655109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe324072410 con 0x7fe3241fda50 2024-04-24T22:39:47.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.965+0000 7fe329a00640 1 -- 172.21.15.47:0/4135655109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3241fcf70 con 0x7fe3241fda50 2024-04-24T22:39:47.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.967+0000 7fc48f200640 1 Processor -- start 2024-04-24T22:39:47.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.965+0000 7fd350c00640 1 -- 172.21.15.47:0/2483301724 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd33c01a070 con 0x7fd354073280 2024-04-24T22:39:47.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.966+0000 7fd350c00640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd33006c060 0x7fd33006e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.967+0000 7fd350c00640 1 -- 172.21.15.47:0/2483301724 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd33c08a860 con 0x7fd354073280 2024-04-24T22:39:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.968+0000 7fc0af400640 1 -- 172.21.15.47:0/1972396272 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0a8059770 con 0x7fc0b80cc660 2024-04-24T22:39:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.968+0000 7fc48f200640 1 -- start start 2024-04-24T22:39:47.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.968+0000 7fd353e00640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd33006c060 0x7fd33006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.967+0000 7fe329a00640 1 -- 172.21.15.47:0/4135655109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe324073b90 con 0x7fe3241fda50 2024-04-24T22:39:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.968+0000 7fe31aa00640 1 -- 172.21.15.47:0/4135655109 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe31001a070 con 0x7fe3241fda50 2024-04-24T22:39:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.969+0000 7fc48f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc488046520 0x7fc4880bd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.969+0000 7fc48f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc48804beb0 0x7fc4880bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fd353e00640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd33006c060 0x7fd33006e520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd34c02a2b0 tx=0x7fd34c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.969+0000 7fc48de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc488046520 0x7fc4880bd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc488046520 0x7fc4880bd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54092/0 (socket says 172.21.15.47:54092) 2024-04-24T22:39:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48de00640 1 -- 172.21.15.47:0/3156349480 learned_addr learned my addr 172.21.15.47:0/3156349480 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48f200640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4880dd4e0 0x7fc4880ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48d400640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc48804beb0 0x7fc4880bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48f200640 1 -- 172.21.15.47:0/3156349480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880e3d80 con 0x7fc488046520 2024-04-24T22:39:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48f200640 1 -- 172.21.15.47:0/3156349480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880e3aa0 con 0x7fc48804beb0 2024-04-24T22:39:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48f200640 1 -- 172.21.15.47:0/3156349480 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc488046000 con 0x7fc4880dd4e0 2024-04-24T22:39:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48e800640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4880dd4e0 0x7fc4880ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48d400640 1 -- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4880dd4e0 msgr2=0x7fc4880ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48d400640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4880dd4e0 0x7fc4880ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48d400640 1 -- 172.21.15.47:0/3156349480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc488046520 msgr2=0x7fc4880bd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48d400640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc488046520 0x7fc4880bd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.970+0000 7fc48d400640 1 -- 172.21.15.47:0/3156349480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4880be100 con 0x7fc48804beb0 2024-04-24T22:39:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.971+0000 7fc48de00640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc488046520 0x7fc4880bd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:47.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.971+0000 7fc48d400640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc48804beb0 0x7fc4880bd7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc48800e240 tx=0x7fc47c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.971+0000 7fe31aa00640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe30406be60 0x7fe30406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.971+0000 7fe31aa00640 1 -- 172.21.15.47:0/4135655109 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe31008a0b0 con 0x7fe3241fda50 2024-04-24T22:39:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.972+0000 7fe322a00640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe30406be60 0x7fe30406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.972+0000 7fd350c00640 1 -- 172.21.15.47:0/2483301724 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd3541fabb0 con 0x7fd354073280 2024-04-24T22:39:47.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.973+0000 7fe322a00640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe30406be60 0x7fe30406e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe324070600 tx=0x7fe31c012040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.973+0000 7fc486000640 1 -- 172.21.15.47:0/3156349480 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c03f020 con 0x7fc48804beb0 2024-04-24T22:39:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.973+0000 7fc486000640 1 -- 172.21.15.47:0/3156349480 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47c037040 con 0x7fc48804beb0 2024-04-24T22:39:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.973+0000 7fc486000640 1 -- 172.21.15.47:0/3156349480 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c036070 con 0x7fc48804beb0 2024-04-24T22:39:47.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.975+0000 7fe31aa00640 1 -- 172.21.15.47:0/4135655109 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe3100582e0 con 0x7fe3241fda50 2024-04-24T22:39:47.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.977+0000 7fc48f200640 1 -- 172.21.15.47:0/3156349480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4880bde20 con 0x7fc48804beb0 2024-04-24T22:39:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.977+0000 7fc48f200640 1 -- 172.21.15.47:0/3156349480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48804a440 con 0x7fc48804beb0 2024-04-24T22:39:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.979+0000 7fc47b400640 1 -- 172.21.15.47:0/3156349480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc464005290 con 0x7fc48804beb0 2024-04-24T22:39:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.985+0000 7fc486000640 1 -- 172.21.15.47:0/3156349480 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc47c042020 con 0x7fc48804beb0 2024-04-24T22:39:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.987+0000 7fc486000640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc47006bf00 0x7fc47006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.987+0000 7fc486000640 1 -- 172.21.15.47:0/3156349480 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc47c0ab510 con 0x7fc48804beb0 2024-04-24T22:39:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.988+0000 7fc486000640 1 -- 172.21.15.47:0/3156349480 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc47c049030 con 0x7fc48804beb0 2024-04-24T22:39:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.988+0000 7fc48de00640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc47006bf00 0x7fc47006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:46.989+0000 7fc48de00640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc47006bf00 0x7fc47006e3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc480007fd0 tx=0x7fc480007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.364+0000 7f6973e00640 1 -- 172.21.15.47:0/2387203116 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f696c0c4da0 con 0x7f696c0c5740 2024-04-24T22:39:47.368 INFO:teuthology.orchestra.run.smithi047.stdout:55834574855 2024-04-24T22:39:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.365+0000 7f6969600640 1 -- 172.21.15.47:0/2387203116 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f69580794e0 con 0x7f696c0c5740 2024-04-24T22:39:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.369+0000 7f6973e00640 1 -- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f694c06bf00 msgr2=0x7f694c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.369+0000 7f6973e00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f694c06bf00 0x7f694c06e3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f696000f290 tx=0x7f6960009040 comp rx=0 tx=0).stop 2024-04-24T22:39:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.369+0000 7f6973e00640 1 -- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f696c0c5740 msgr2=0x7f696c0cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.369+0000 7f6973e00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f696c0c5740 0x7f696c0cf1d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6958031000 tx=0x7f69580046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.370+0000 7f6973e00640 1 -- 172.21.15.47:0/2387203116 shutdown_connections 2024-04-24T22:39:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.370+0000 7f6973e00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f694c06bf00 0x7f694c06e3c0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f696000f290 tx=0x7f6960009040 comp rx=0 tx=0).stop 2024-04-24T22:39:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.370+0000 7f6973e00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f696c16b450 0x7f696c0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.370+0000 7f6973e00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f696c0c5740 0x7f696c0cf1d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.370+0000 7f6973e00640 1 --2- 172.21.15.47:0/2387203116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f696c0b4fa0 0x7f696c1fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.370+0000 7f6973e00640 1 -- 172.21.15.47:0/2387203116 >> 172.21.15.47:0/2387203116 conn(0x7f696c0d0270 msgr2=0x7f696c0cde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.370+0000 7f6973e00640 1 -- 172.21.15.47:0/2387203116 shutdown_connections 2024-04-24T22:39:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.370+0000 7f6973e00640 1 -- 172.21.15.47:0/2387203116 wait complete. 2024-04-24T22:39:47.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-24T22:39:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.397+0000 7fa493600640 1 -- 172.21.15.47:0/2005417635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa48c0b2de0 con 0x7fa48c0cc840 2024-04-24T22:39:47.399 DEBUG:teuthology.parallel:result is None 2024-04-24T22:39:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.397+0000 7fa488c00640 1 -- 172.21.15.47:0/2005417635 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa484059660 con 0x7fa48c0cc840 2024-04-24T22:39:47.400 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-04-24T22:39:47.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.398+0000 7ff635c00640 1 -- 172.21.15.47:0/115830144 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff6300c76d0 con 0x7ff630158be0 2024-04-24T22:39:47.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.399+0000 7ff626a00640 1 -- 172.21.15.47:0/115830144 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff61c079a30 con 0x7ff630158be0 2024-04-24T22:39:47.400 INFO:teuthology.orchestra.run.smithi047.stdout:55834574855 2024-04-24T22:39:47.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.401+0000 7fa493600640 1 -- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa46c06be80 msgr2=0x7fa46c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.401+0000 7fa493600640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa46c06be80 0x7fa46c06e340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa47800c900 tx=0x7fa478008040 comp rx=0 tx=0).stop 2024-04-24T22:39:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.401+0000 7fa493600640 1 -- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0cc840 msgr2=0x7fa48c0c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.401+0000 7fa493600640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0cc840 0x7fa48c0c9300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c7b0 tx=0x7fa48400cc70 comp rx=0 tx=0).stop 2024-04-24T22:39:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.401+0000 7fa493600640 1 -- 172.21.15.47:0/2005417635 shutdown_connections 2024-04-24T22:39:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.401+0000 7fa493600640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa46c06be80 0x7fa46c06e340 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.401+0000 7fa493600640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa48c0cc840 0x7fa48c0c9300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.401+0000 7fa493600640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa48c0cb6d0 0x7fa48c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.401+0000 7fa493600640 1 --2- 172.21.15.47:0/2005417635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa48c073380 0x7fa48c0c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.401+0000 7fa493600640 1 -- 172.21.15.47:0/2005417635 >> 172.21.15.47:0/2005417635 conn(0x7fa48c06c940 msgr2=0x7fa48c1e5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.402+0000 7fa493600640 1 -- 172.21.15.47:0/2005417635 shutdown_connections 2024-04-24T22:39:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.402+0000 7fa493600640 1 -- 172.21.15.47:0/2005417635 wait complete. 2024-04-24T22:39:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.402+0000 7fd852000640 1 -- 172.21.15.47:0/3163830156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd834005580 con 0x7fd8600c6b00 2024-04-24T22:39:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7fd85cc00640 1 -- 172.21.15.47:0/3163830156 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd84c079a20 con 0x7fd8600c6b00 2024-04-24T22:39:47.404 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-04-24T22:39:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 -- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff61006be60 msgr2=0x7ff61006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff61006be60 0x7ff61006e320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff620007d50 tx=0x7ff620007ca0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 -- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff630158be0 msgr2=0x7ff63015aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff630158be0 0x7ff63015aab0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff61c031000 tx=0x7ff61c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 -- 172.21.15.47:0/115830144 shutdown_connections 2024-04-24T22:39:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff61006be60 0x7ff61006e320 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff630158fc0 0x7ff63015aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff630158be0 0x7ff63015aab0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 --2- 172.21.15.47:0/115830144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6300c1280 0x7ff630161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 -- 172.21.15.47:0/115830144 >> 172.21.15.47:0/115830144 conn(0x7ff63006e580 msgr2=0x7ff63006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 -- 172.21.15.47:0/115830144 shutdown_connections 2024-04-24T22:39:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.403+0000 7ff635c00640 1 -- 172.21.15.47:0/115830144 wait complete. 2024-04-24T22:39:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.406+0000 7fd852000640 1 -- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd84006be30 msgr2=0x7fd84006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.406+0000 7fd852000640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd84006be30 0x7fd84006e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd854005f60 tx=0x7fd854000f10 comp rx=0 tx=0).stop 2024-04-24T22:39:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.406+0000 7fd852000640 1 -- 172.21.15.47:0/3163830156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8600c6b00 msgr2=0x7fd86015b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.406+0000 7fd852000640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8600c6b00 0x7fd86015b9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd84c031000 tx=0x7fd84c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.407+0000 7fd852000640 1 -- 172.21.15.47:0/3163830156 shutdown_connections 2024-04-24T22:39:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.407+0000 7fd852000640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd84006be30 0x7fd84006e2f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.407+0000 7fd852000640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8601673d0 0x7fd86015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.407+0000 7fd852000640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8600c6b00 0x7fd86015b9f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.407+0000 7fd852000640 1 --2- 172.21.15.47:0/3163830156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd860071de0 0x7fd86015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.407+0000 7fd852000640 1 -- 172.21.15.47:0/3163830156 >> 172.21.15.47:0/3163830156 conn(0x7fd86006e580 msgr2=0x7fd8600727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.407+0000 7fd852000640 1 -- 172.21.15.47:0/3163830156 shutdown_connections 2024-04-24T22:39:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.407+0000 7fd852000640 1 -- 172.21.15.47:0/3163830156 wait complete. 2024-04-24T22:39:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.410+0000 7fd35a000640 1 -- 172.21.15.47:0/2483301724 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd32c005580 con 0x7fd354073280 2024-04-24T22:39:47.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.411+0000 7fc0bce00640 1 -- 172.21.15.47:0/1972396272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc0b804f5e0 con 0x7fc0b80cc660 2024-04-24T22:39:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.411+0000 7fd350c00640 1 -- 172.21.15.47:0/2483301724 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd33c058a90 con 0x7fd354073280 2024-04-24T22:39:47.413 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-04-24T22:39:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.411+0000 7fc0af400640 1 -- 172.21.15.47:0/1972396272 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc0a8059070 con 0x7fc0b80cc660 2024-04-24T22:39:47.415 INFO:teuthology.orchestra.run.smithi047.stdout:55834574855 2024-04-24T22:39:47.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.416+0000 7fc0bce00640 1 -- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc09406be80 msgr2=0x7fc09406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.416+0000 7fd35a000640 1 -- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd33006c060 msgr2=0x7fd33006e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.416+0000 7fc0bce00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc09406be80 0x7fc09406e340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0034000 tx=0x7fc0a002e000 comp rx=0 tx=0).stop 2024-04-24T22:39:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fd35a000640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd33006c060 0x7fd33006e520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd34c02a2b0 tx=0x7fd34c02e000 comp rx=0 tx=0).stop 2024-04-24T22:39:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fd35a000640 1 -- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 msgr2=0x7fd354151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fd35a000640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd354151130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00e870 tx=0x7fd33c00ed30 comp rx=0 tx=0).stop 2024-04-24T22:39:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fd35a000640 1 -- 172.21.15.47:0/2483301724 shutdown_connections 2024-04-24T22:39:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fc0bce00640 1 -- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0b80cc660 msgr2=0x7fc0b81495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fc0bce00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0b80cc660 0x7fc0b81495b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800c7f0 tx=0x7fc0a800ccb0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fd35a000640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd33006c060 0x7fd33006e520 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fd35a000640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd354154560 0x7fd354200e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fd35a000640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3540cc660 0x7fd3541fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fd35a000640 1 --2- 172.21.15.47:0/2483301724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd354073280 0x7fd354151130 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fd35a000640 1 -- 172.21.15.47:0/2483301724 >> 172.21.15.47:0/2483301724 conn(0x7fd3540672c0 msgr2=0x7fd3540c4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fc0bce00640 1 -- 172.21.15.47:0/1972396272 shutdown_connections 2024-04-24T22:39:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fc0bce00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc09406be80 0x7fc09406e340 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fc0bce00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0b8154560 0x7fc0b81522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fc0bce00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0b80cc660 0x7fc0b81495b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fc0bce00640 1 --2- 172.21.15.47:0/1972396272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0b8073280 0x7fc0b81439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fc0bce00640 1 -- 172.21.15.47:0/1972396272 >> 172.21.15.47:0/1972396272 conn(0x7fc0b80672c0 msgr2=0x7fc0b8153480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fd35a000640 1 -- 172.21.15.47:0/2483301724 shutdown_connections 2024-04-24T22:39:47.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fc0bce00640 1 -- 172.21.15.47:0/1972396272 shutdown_connections 2024-04-24T22:39:47.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.417+0000 7fc0bce00640 1 -- 172.21.15.47:0/1972396272 wait complete. 2024-04-24T22:39:47.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.418+0000 7fd35a000640 1 -- 172.21.15.47:0/2483301724 wait complete. 2024-04-24T22:39:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.424+0000 7fe329a00640 1 -- 172.21.15.47:0/4135655109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe32406c200 con 0x7fe3241fda50 2024-04-24T22:39:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.425+0000 7fe31aa00640 1 -- 172.21.15.47:0/4135655109 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe31005c5e0 con 0x7fe3241fda50 2024-04-24T22:39:47.426 INFO:teuthology.orchestra.run.smithi047.stdout:55834574854 2024-04-24T22:39:47.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-04-24T22:39:47.429 DEBUG:teuthology.parallel:result is None 2024-04-24T22:39:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.428+0000 7fe329a00640 1 -- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe30406be60 msgr2=0x7fe30406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.428+0000 7fe329a00640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe30406be60 0x7fe30406e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe324070600 tx=0x7fe31c012040 comp rx=0 tx=0).stop 2024-04-24T22:39:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.428+0000 7fe329a00640 1 -- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3241fda50 msgr2=0x7fe324070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.428+0000 7fe329a00640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3241fda50 0x7fe324070890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe31000c480 tx=0x7fe31000c880 comp rx=0 tx=0).stop 2024-04-24T22:39:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.428+0000 7fe329a00640 1 -- 172.21.15.47:0/4135655109 shutdown_connections 2024-04-24T22:39:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.428+0000 7fe329a00640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe30406be60 0x7fe30406e320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.428+0000 7fe329a00640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe324153420 0x7fe3240c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.428+0000 7fe329a00640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3241fda50 0x7fe324070890 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.428+0000 7fe329a00640 1 --2- 172.21.15.47:0/4135655109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3080051c0 0x7fe324071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.428+0000 7fe329a00640 1 -- 172.21.15.47:0/4135655109 >> 172.21.15.47:0/4135655109 conn(0x7fe32406e580 msgr2=0x7fe3241576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.429+0000 7fe329a00640 1 -- 172.21.15.47:0/4135655109 shutdown_connections 2024-04-24T22:39:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.429+0000 7fe329a00640 1 -- 172.21.15.47:0/4135655109 wait complete. 2024-04-24T22:39:47.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-24T22:39:47.434 DEBUG:teuthology.parallel:result is None 2024-04-24T22:39:47.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.435+0000 7fc47b400640 1 -- 172.21.15.47:0/3156349480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc464005850 con 0x7fc48804beb0 2024-04-24T22:39:47.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.435+0000 7fc486000640 1 -- 172.21.15.47:0/3156349480 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc47c079e10 con 0x7fc48804beb0 2024-04-24T22:39:47.437 INFO:teuthology.orchestra.run.smithi047.stdout:55834574855 2024-04-24T22:39:47.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-04-24T22:39:47.438 DEBUG:teuthology.parallel:result is None 2024-04-24T22:39:47.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 -- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc47006bf00 msgr2=0x7fc47006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc47006bf00 0x7fc47006e3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc480007fd0 tx=0x7fc480007de0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 -- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc48804beb0 msgr2=0x7fc4880bd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc48804beb0 0x7fc4880bd7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc48800e240 tx=0x7fc47c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 -- 172.21.15.47:0/3156349480 shutdown_connections 2024-04-24T22:39:47.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc47006bf00 0x7fc47006e3c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4880dd4e0 0x7fc4880ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc48804beb0 0x7fc4880bd7c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 --2- 172.21.15.47:0/3156349480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc488046520 0x7fc4880bd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 -- 172.21.15.47:0/3156349480 >> 172.21.15.47:0/3156349480 conn(0x7fc4880505d0 msgr2=0x7fc48804f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 -- 172.21.15.47:0/3156349480 shutdown_connections 2024-04-24T22:39:47.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.438+0000 7fc47b400640 1 -- 172.21.15.47:0/3156349480 wait complete. 2024-04-24T22:39:47.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-04-24T22:39:47.445 DEBUG:teuthology.parallel:result is None 2024-04-24T22:39:47.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-24T22:39:47.447 DEBUG:teuthology.parallel:result is None 2024-04-24T22:39:47.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-04-24T22:39:47.450 DEBUG:teuthology.parallel:result is None 2024-04-24T22:39:47.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-24T22:39:47.456 DEBUG:teuthology.parallel:result is None 2024-04-24T22:39:47.456 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T22:39:47.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:39:47.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.693+0000 7ffa19c00640 1 Processor -- start 2024-04-24T22:39:47.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.693+0000 7ffa19c00640 1 -- start start 2024-04-24T22:39:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.694+0000 7ffa19c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa14154560 0x7ffa140cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.694+0000 7ffa19c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa140cbdc0 con 0x7ffa140cb970 2024-04-24T22:39:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.694+0000 7ffa19c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1415b2c0 con 0x7ffa14073280 2024-04-24T22:39:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.694+0000 7ffa19c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1415da90 con 0x7ffa14154560 2024-04-24T22:39:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.695+0000 7ffa19200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffa140cb970 0x7ffa140ccd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40930/0 (socket says 172.21.15.47:40930) 2024-04-24T22:39:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.695+0000 7ffa19200640 1 -- 172.21.15.47:0/2416227382 learned_addr learned my addr 172.21.15.47:0/2416227382 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.695+0000 7ffa13e00640 1 --2- 172.21.15.47:0/2416227382 >> v2:172.21.15.113:3300/0 conn(0x7ffa14154560 0x7ffa140cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.695+0000 7ffa13400640 1 -- 172.21.15.47:0/2416227382 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4133433618 0 0) 0x7ffa140cbdc0 con 0x7ffa140cb970 2024-04-24T22:39:47.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.695+0000 7ffa13400640 1 -- 172.21.15.47:0/2416227382 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9f8003520 con 0x7ffa140cb970 2024-04-24T22:39:47.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.695+0000 7ffa13400640 1 -- 172.21.15.47:0/2416227382 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106507256 0 0) 0x7ffa1415b2c0 con 0x7ffa14073280 2024-04-24T22:39:47.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.695+0000 7ffa13400640 1 -- 172.21.15.47:0/2416227382 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa140cbdc0 con 0x7ffa14073280 2024-04-24T22:39:47.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.695+0000 7ffa13e00640 1 -- 172.21.15.47:0/2416227382 >> v1:172.21.15.113:6789/0 conn(0x7ffa14073280 legacy=0x7ffa140cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.695+0000 7ffa13e00640 1 -- 172.21.15.47:0/2416227382 >> v1:172.21.15.47:6789/0 conn(0x7ffa140cb970 legacy=0x7ffa140ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.695+0000 7ffa13e00640 1 -- 172.21.15.47:0/2416227382 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1415dd70 con 0x7ffa14154560 2024-04-24T22:39:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.697+0000 7ffa13e00640 1 --2- 172.21.15.47:0/2416227382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa14154560 0x7ffa140cb430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa0402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb8d496ba36c9cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.697+0000 7ffa13400640 1 -- 172.21.15.47:0/2416227382 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04037070 con 0x7ffa14154560 2024-04-24T22:39:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.697+0000 7ffa13400640 1 -- 172.21.15.47:0/2416227382 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa04032440 con 0x7ffa14154560 2024-04-24T22:39:47.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.697+0000 7ffa13400640 1 -- 172.21.15.47:0/2416227382 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04032810 con 0x7ffa14154560 2024-04-24T22:39:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.697+0000 7ffa19c00640 1 -- 172.21.15.47:0/2416227382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa14154560 msgr2=0x7ffa140cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.697+0000 7ffa19c00640 1 --2- 172.21.15.47:0/2416227382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa14154560 0x7ffa140cb430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa0402a230 comp rx=0 tx=0).stop 2024-04-24T22:39:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.698+0000 7ffa19c00640 1 -- 172.21.15.47:0/2416227382 shutdown_connections 2024-04-24T22:39:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.698+0000 7ffa19c00640 1 --2- 172.21.15.47:0/2416227382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa14154560 0x7ffa140cb430 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.698+0000 7ffa19c00640 1 -- 172.21.15.47:0/2416227382 >> 172.21.15.47:0/2416227382 conn(0x7ffa140672c0 msgr2=0x7ffa14072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.698+0000 7ffa19c00640 1 -- 172.21.15.47:0/2416227382 shutdown_connections 2024-04-24T22:39:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.698+0000 7ffa19c00640 1 -- 172.21.15.47:0/2416227382 wait complete. 2024-04-24T22:39:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.700+0000 7ffa19c00640 1 Processor -- start 2024-04-24T22:39:47.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.700+0000 7ffa19c00640 1 -- start start 2024-04-24T22:39:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.701+0000 7ffa19c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa14073280 0x7ffa140cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.701+0000 7ffa19c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa140cb970 0x7ffa140ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.701+0000 7ffa19c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa14154560 0x7ffa141f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.701+0000 7ffa19c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffa140caad0 con 0x7ffa140cb970 2024-04-24T22:39:47.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.701+0000 7ffa19c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1415e330 con 0x7ffa14073280 2024-04-24T22:39:47.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.701+0000 7ffa19c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1415ae80 con 0x7ffa14154560 2024-04-24T22:39:47.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa140cb970 0x7ffa140ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa140cb970 0x7ffa140ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54096/0 (socket says 172.21.15.47:54096) 2024-04-24T22:39:47.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa0be00640 1 -- 172.21.15.47:0/2740767955 learned_addr learned my addr 172.21.15.47:0/2740767955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:47.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa13e00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa14073280 0x7ffa140cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa19200640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa14154560 0x7ffa141f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa0be00640 1 -- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa14154560 msgr2=0x7ffa141f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa0be00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa14154560 0x7ffa141f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa0be00640 1 -- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa14073280 msgr2=0x7ffa140cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa0be00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa14073280 0x7ffa140cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa0be00640 1 -- 172.21.15.47:0/2740767955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa140cdcd0 con 0x7ffa140cb970 2024-04-24T22:39:47.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.702+0000 7ffa13e00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa14073280 0x7ffa140cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:47.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.703+0000 7ffa0be00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa140cb970 0x7ffa140ccba0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00c460 tx=0x7ff9fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.703+0000 7ffa10c00640 1 -- 172.21.15.47:0/2740767955 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc017070 con 0x7ffa140cb970 2024-04-24T22:39:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.703+0000 7ffa10c00640 1 -- 172.21.15.47:0/2740767955 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9fc0052f0 con 0x7ffa140cb970 2024-04-24T22:39:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.703+0000 7ffa10c00640 1 -- 172.21.15.47:0/2740767955 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc003830 con 0x7ffa140cb970 2024-04-24T22:39:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.703+0000 7ffa19c00640 1 -- 172.21.15.47:0/2740767955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa140cdfb0 con 0x7ffa140cb970 2024-04-24T22:39:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.703+0000 7ffa19c00640 1 -- 172.21.15.47:0/2740767955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa140ce570 con 0x7ffa140cb970 2024-04-24T22:39:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.704+0000 7ffa19c00640 1 -- 172.21.15.47:0/2740767955 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa140c6080 con 0x7ffa140cb970 2024-04-24T22:39:47.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.705+0000 7ffa10c00640 1 -- 172.21.15.47:0/2740767955 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9fc01a070 con 0x7ffa140cb970 2024-04-24T22:39:47.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.706+0000 7ffa10c00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff9f006be30 0x7ff9f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:47.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.706+0000 7ffa10c00640 1 -- 172.21.15.47:0/2740767955 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff9fc08a2c0 con 0x7ffa140cb970 2024-04-24T22:39:47.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.706+0000 7ffa13e00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff9f006be30 0x7ff9f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:47.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.707+0000 7ffa13e00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff9f006be30 0x7ff9f006e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffa04034000 tx=0x7ffa0402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:47.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.710+0000 7ffa10c00640 1 -- 172.21.15.47:0/2740767955 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9fc0584f0 con 0x7ffa140cb970 2024-04-24T22:39:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.902+0000 7ffa19c00640 1 -- 172.21.15.47:0/2740767955 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa1404f5e0 con 0x7ff9f006be30 2024-04-24T22:39:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.909+0000 7ffa10c00640 1 -- 172.21.15.47:0/2740767955 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251835 (secure 0 0 0) 0x7ffa1404f5e0 con 0x7ff9f006be30 2024-04-24T22:39:47.916 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:47.918 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:39:47.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 -- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff9f006be30 msgr2=0x7ff9f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff9f006be30 0x7ff9f006e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffa04034000 tx=0x7ffa0402e000 comp rx=0 tx=0).stop 2024-04-24T22:39:47.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 -- 172.21.15.47:0/2740767955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa140cb970 msgr2=0x7ffa140ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa140cb970 0x7ffa140ccba0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00c460 tx=0x7ff9fc00c920 comp rx=0 tx=0).stop 2024-04-24T22:39:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 -- 172.21.15.47:0/2740767955 shutdown_connections 2024-04-24T22:39:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff9f006be30 0x7ff9f006e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa14154560 0x7ffa141f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa140cb970 0x7ffa140ccba0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 --2- 172.21.15.47:0/2740767955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa14073280 0x7ffa140cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 -- 172.21.15.47:0/2740767955 >> 172.21.15.47:0/2740767955 conn(0x7ffa140672c0 msgr2=0x7ffa14153370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 -- 172.21.15.47:0/2740767955 shutdown_connections 2024-04-24T22:39:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:47.915+0000 7ffa19c00640 1 -- 172.21.15.47:0/2740767955 wait complete. 2024-04-24T22:39:47.934 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-24T22:39:46.643666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235688,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739032,"statfs":{"total":773094113280,"available":772852768768,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.969689"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706276+0000","last_change":"2024-04-24T22:39:37.699192+0000","last_active":"2024-04-24T22:39:38.706276+0000","last_peered":"2024-04-24T22:39:38.706276+0000","last_clean":"2024-04-24T22:39:38.706276+0000","last_became_active":"2024-04-24T22:39:37.698920+0000","last_became_peered":"2024-04-24T22:39:37.698920+0000","last_unstale":"2024-04-24T22:39:38.706276+0000","last_undegraded":"2024-04-24T22:39:38.706276+0000","last_fullsized":"2024-04-24T22:39:38.706276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.821236+0000","last_change":"2024-04-24T22:39:37.697245+0000","last_active":"2024-04-24T22:39:38.821236+0000","last_peered":"2024-04-24T22:39:38.821236+0000","last_clean":"2024-04-24T22:39:38.821236+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:39:38.821236+0000","last_undegraded":"2024-04-24T22:39:38.821236+0000","last_fullsized":"2024-04-24T22:39:38.821236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705212+0000","last_change":"2024-04-24T22:39:37.709539+0000","last_active":"2024-04-24T22:39:38.705212+0000","last_peered":"2024-04-24T22:39:38.705212+0000","last_clean":"2024-04-24T22:39:38.705212+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:39:38.705212+0000","last_undegraded":"2024-04-24T22:39:38.705212+0000","last_fullsized":"2024-04-24T22:39:38.705212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820523+0000","last_change":"2024-04-24T22:39:37.719136+0000","last_active":"2024-04-24T22:39:38.820523+0000","last_peered":"2024-04-24T22:39:38.820523+0000","last_clean":"2024-04-24T22:39:38.820523+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:39:38.820523+0000","last_undegraded":"2024-04-24T22:39:38.820523+0000","last_fullsized":"2024-04-24T22:39:38.820523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705940+0000","last_change":"2024-04-24T22:39:37.704671+0000","last_active":"2024-04-24T22:39:38.705940+0000","last_peered":"2024-04-24T22:39:38.705940+0000","last_clean":"2024-04-24T22:39:38.705940+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:39:38.705940+0000","last_undegraded":"2024-04-24T22:39:38.705940+0000","last_fullsized":"2024-04-24T22:39:38.705940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820279+0000","last_change":"2024-04-24T22:39:37.703880+0000","last_active":"2024-04-24T22:39:38.820279+0000","last_peered":"2024-04-24T22:39:38.820279+0000","last_clean":"2024-04-24T22:39:38.820279+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:39:38.820279+0000","last_undegraded":"2024-04-24T22:39:38.820279+0000","last_fullsized":"2024-04-24T22:39:38.820279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705748+0000","last_change":"2024-04-24T22:39:37.703539+0000","last_active":"2024-04-24T22:39:38.705748+0000","last_peered":"2024-04-24T22:39:38.705748+0000","last_clean":"2024-04-24T22:39:38.705748+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:39:38.705748+0000","last_undegraded":"2024-04-24T22:39:38.705748+0000","last_fullsized":"2024-04-24T22:39:38.705748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707950+0000","last_change":"2024-04-24T22:39:37.713543+0000","last_active":"2024-04-24T22:39:38.707950+0000","last_peered":"2024-04-24T22:39:38.707950+0000","last_clean":"2024-04-24T22:39:38.707950+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:39:38.707950+0000","last_undegraded":"2024-04-24T22:39:38.707950+0000","last_fullsized":"2024-04-24T22:39:38.707950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706353+0000","last_change":"2024-04-24T22:39:37.706715+0000","last_active":"2024-04-24T22:39:38.706353+0000","last_peered":"2024-04-24T22:39:38.706353+0000","last_clean":"2024-04-24T22:39:38.706353+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:39:38.706353+0000","last_undegraded":"2024-04-24T22:39:38.706353+0000","last_fullsized":"2024-04-24T22:39:38.706353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819558+0000","last_change":"2024-04-24T22:39:37.701250+0000","last_active":"2024-04-24T22:39:38.819558+0000","last_peered":"2024-04-24T22:39:38.819558+0000","last_clean":"2024-04-24T22:39:38.819558+0000","last_became_active":"2024-04-24T22:39:37.700959+0000","last_became_peered":"2024-04-24T22:39:37.700959+0000","last_unstale":"2024-04-24T22:39:38.819558+0000","last_undegraded":"2024-04-24T22:39:38.819558+0000","last_fullsized":"2024-04-24T22:39:38.819558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819186+0000","last_change":"2024-04-24T22:39:37.701069+0000","last_active":"2024-04-24T22:39:38.819186+0000","last_peered":"2024-04-24T22:39:38.819186+0000","last_clean":"2024-04-24T22:39:38.819186+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:39:38.819186+0000","last_undegraded":"2024-04-24T22:39:38.819186+0000","last_fullsized":"2024-04-24T22:39:38.819186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819923+0000","last_change":"2024-04-24T22:39:37.693508+0000","last_active":"2024-04-24T22:39:38.819923+0000","last_peered":"2024-04-24T22:39:38.819923+0000","last_clean":"2024-04-24T22:39:38.819923+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:39:38.819923+0000","last_undegraded":"2024-04-24T22:39:38.819923+0000","last_fullsized":"2024-04-24T22:39:38.819923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708270+0000","last_change":"2024-04-24T22:39:37.705157+0000","last_active":"2024-04-24T22:39:38.708270+0000","last_peered":"2024-04-24T22:39:38.708270+0000","last_clean":"2024-04-24T22:39:38.708270+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:39:38.708270+0000","last_undegraded":"2024-04-24T22:39:38.708270+0000","last_fullsized":"2024-04-24T22:39:38.708270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704755+0000","last_change":"2024-04-24T22:39:37.698333+0000","last_active":"2024-04-24T22:39:38.704755+0000","last_peered":"2024-04-24T22:39:38.704755+0000","last_clean":"2024-04-24T22:39:38.704755+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:39:38.704755+0000","last_undegraded":"2024-04-24T22:39:38.704755+0000","last_fullsized":"2024-04-24T22:39:38.704755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:57.510197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820219+0000","last_change":"2024-04-24T22:39:37.696874+0000","last_active":"2024-04-24T22:39:38.820219+0000","last_peered":"2024-04-24T22:39:38.820219+0000","last_clean":"2024-04-24T22:39:38.820219+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:39:38.820219+0000","last_undegraded":"2024-04-24T22:39:38.820219+0000","last_fullsized":"2024-04-24T22:39:38.820219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820021+0000","last_change":"2024-04-24T22:39:37.703446+0000","last_active":"2024-04-24T22:39:38.820021+0000","last_peered":"2024-04-24T22:39:38.820021+0000","last_clean":"2024-04-24T22:39:38.820021+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:39:38.820021+0000","last_undegraded":"2024-04-24T22:39:38.820021+0000","last_fullsized":"2024-04-24T22:39:38.820021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820971+0000","last_change":"2024-04-24T22:39:37.701515+0000","last_active":"2024-04-24T22:39:38.820971+0000","last_peered":"2024-04-24T22:39:38.820971+0000","last_clean":"2024-04-24T22:39:38.820971+0000","last_became_active":"2024-04-24T22:39:37.700439+0000","last_became_peered":"2024-04-24T22:39:37.700439+0000","last_unstale":"2024-04-24T22:39:38.820971+0000","last_undegraded":"2024-04-24T22:39:38.820971+0000","last_fullsized":"2024-04-24T22:39:38.820971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819628+0000","last_change":"2024-04-24T22:39:37.703899+0000","last_active":"2024-04-24T22:39:38.819628+0000","last_peered":"2024-04-24T22:39:38.819628+0000","last_clean":"2024-04-24T22:39:38.819628+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:39:38.819628+0000","last_undegraded":"2024-04-24T22:39:38.819628+0000","last_fullsized":"2024-04-24T22:39:38.819628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706244+0000","last_change":"2024-04-24T22:39:37.711019+0000","last_active":"2024-04-24T22:39:38.706244+0000","last_peered":"2024-04-24T22:39:38.706244+0000","last_clean":"2024-04-24T22:39:38.706244+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:39:38.706244+0000","last_undegraded":"2024-04-24T22:39:38.706244+0000","last_fullsized":"2024-04-24T22:39:38.706244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819357+0000","last_change":"2024-04-24T22:39:37.696214+0000","last_active":"2024-04-24T22:39:38.819357+0000","last_peered":"2024-04-24T22:39:38.819357+0000","last_clean":"2024-04-24T22:39:38.819357+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:39:38.819357+0000","last_undegraded":"2024-04-24T22:39:38.819357+0000","last_fullsized":"2024-04-24T22:39:38.819357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705029+0000","last_change":"2024-04-24T22:39:37.700151+0000","last_active":"2024-04-24T22:39:38.705029+0000","last_peered":"2024-04-24T22:39:38.705029+0000","last_clean":"2024-04-24T22:39:38.705029+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:39:38.705029+0000","last_undegraded":"2024-04-24T22:39:38.705029+0000","last_fullsized":"2024-04-24T22:39:38.705029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820267+0000","last_change":"2024-04-24T22:39:37.705083+0000","last_active":"2024-04-24T22:39:38.820267+0000","last_peered":"2024-04-24T22:39:38.820267+0000","last_clean":"2024-04-24T22:39:38.820267+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:39:38.820267+0000","last_undegraded":"2024-04-24T22:39:38.820267+0000","last_fullsized":"2024-04-24T22:39:38.820267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708194+0000","last_change":"2024-04-24T22:39:37.711193+0000","last_active":"2024-04-24T22:39:38.708194+0000","last_peered":"2024-04-24T22:39:38.708194+0000","last_clean":"2024-04-24T22:39:38.708194+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:39:38.708194+0000","last_undegraded":"2024-04-24T22:39:38.708194+0000","last_fullsized":"2024-04-24T22:39:38.708194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820128+0000","last_change":"2024-04-24T22:39:37.713910+0000","last_active":"2024-04-24T22:39:38.820128+0000","last_peered":"2024-04-24T22:39:38.820128+0000","last_clean":"2024-04-24T22:39:38.820128+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:39:38.820128+0000","last_undegraded":"2024-04-24T22:39:38.820128+0000","last_fullsized":"2024-04-24T22:39:38.820128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708266+0000","last_change":"2024-04-24T22:39:37.717191+0000","last_active":"2024-04-24T22:39:38.708266+0000","last_peered":"2024-04-24T22:39:38.708266+0000","last_clean":"2024-04-24T22:39:38.708266+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:39:38.708266+0000","last_undegraded":"2024-04-24T22:39:38.708266+0000","last_fullsized":"2024-04-24T22:39:38.708266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819555+0000","last_change":"2024-04-24T22:39:37.702983+0000","last_active":"2024-04-24T22:39:38.819555+0000","last_peered":"2024-04-24T22:39:38.819555+0000","last_clean":"2024-04-24T22:39:38.819555+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:39:38.819555+0000","last_undegraded":"2024-04-24T22:39:38.819555+0000","last_fullsized":"2024-04-24T22:39:38.819555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820583+0000","last_change":"2024-04-24T22:39:37.714111+0000","last_active":"2024-04-24T22:39:38.820583+0000","last_peered":"2024-04-24T22:39:38.820583+0000","last_clean":"2024-04-24T22:39:38.820583+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:39:38.820583+0000","last_undegraded":"2024-04-24T22:39:38.820583+0000","last_fullsized":"2024-04-24T22:39:38.820583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708923+0000","last_change":"2024-04-24T22:39:37.714877+0000","last_active":"2024-04-24T22:39:38.708923+0000","last_peered":"2024-04-24T22:39:38.708923+0000","last_clean":"2024-04-24T22:39:38.708923+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:39:38.708923+0000","last_undegraded":"2024-04-24T22:39:38.708923+0000","last_fullsized":"2024-04-24T22:39:38.708923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705236+0000","last_change":"2024-04-24T22:39:37.714695+0000","last_active":"2024-04-24T22:39:38.705236+0000","last_peered":"2024-04-24T22:39:38.705236+0000","last_clean":"2024-04-24T22:39:38.705236+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:39:38.705236+0000","last_undegraded":"2024-04-24T22:39:38.705236+0000","last_fullsized":"2024-04-24T22:39:38.705236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820461+0000","last_change":"2024-04-24T22:39:37.719187+0000","last_active":"2024-04-24T22:39:38.820461+0000","last_peered":"2024-04-24T22:39:38.820461+0000","last_clean":"2024-04-24T22:39:38.820461+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:39:38.820461+0000","last_undegraded":"2024-04-24T22:39:38.820461+0000","last_fullsized":"2024-04-24T22:39:38.820461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819723+0000","last_change":"2024-04-24T22:39:37.707283+0000","last_active":"2024-04-24T22:39:38.819723+0000","last_peered":"2024-04-24T22:39:38.819723+0000","last_clean":"2024-04-24T22:39:38.819723+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:39:38.819723+0000","last_undegraded":"2024-04-24T22:39:38.819723+0000","last_fullsized":"2024-04-24T22:39:38.819723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818968+0000","last_change":"2024-04-24T22:39:37.697902+0000","last_active":"2024-04-24T22:39:38.818968+0000","last_peered":"2024-04-24T22:39:38.818968+0000","last_clean":"2024-04-24T22:39:38.818968+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:39:38.818968+0000","last_undegraded":"2024-04-24T22:39:38.818968+0000","last_fullsized":"2024-04-24T22:39:38.818968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819142+0000","last_change":"2024-04-24T22:39:37.705667+0000","last_active":"2024-04-24T22:39:38.819142+0000","last_peered":"2024-04-24T22:39:38.819142+0000","last_clean":"2024-04-24T22:39:38.819142+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:39:38.819142+0000","last_undegraded":"2024-04-24T22:39:38.819142+0000","last_fullsized":"2024-04-24T22:39:38.819142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:50.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.683542+0000","last_change":"2024-04-24T22:39:36.707990+0000","last_active":"2024-04-24T22:39:39.683542+0000","last_peered":"2024-04-24T22:39:39.683542+0000","last_clean":"2024-04-24T22:39:39.683542+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:39:39.683542+0000","last_undegraded":"2024-04-24T22:39:39.683542+0000","last_fullsized":"2024-04-24T22:39:39.683542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706420+0000","last_change":"2024-04-24T22:39:37.700406+0000","last_active":"2024-04-24T22:39:38.706420+0000","last_peered":"2024-04-24T22:39:38.706420+0000","last_clean":"2024-04-24T22:39:38.706420+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:39:38.706420+0000","last_undegraded":"2024-04-24T22:39:38.706420+0000","last_fullsized":"2024-04-24T22:39:38.706420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.675296+0000","last_change":"2024-04-24T22:39:36.701398+0000","last_active":"2024-04-24T22:39:39.675296+0000","last_peered":"2024-04-24T22:39:39.675296+0000","last_clean":"2024-04-24T22:39:39.675296+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:39:39.675296+0000","last_undegraded":"2024-04-24T22:39:39.675296+0000","last_fullsized":"2024-04-24T22:39:39.675296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708466+0000","last_change":"2024-04-24T22:39:36.712294+0000","last_active":"2024-04-24T22:39:38.708466+0000","last_peered":"2024-04-24T22:39:38.708466+0000","last_clean":"2024-04-24T22:39:38.708466+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:39:38.708466+0000","last_undegraded":"2024-04-24T22:39:38.708466+0000","last_fullsized":"2024-04-24T22:39:38.708466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707809+0000","last_change":"2024-04-24T22:39:37.716920+0000","last_active":"2024-04-24T22:39:38.707809+0000","last_peered":"2024-04-24T22:39:38.707809+0000","last_clean":"2024-04-24T22:39:38.707809+0000","last_became_active":"2024-04-24T22:39:37.716641+0000","last_became_peered":"2024-04-24T22:39:37.716641+0000","last_unstale":"2024-04-24T22:39:38.707809+0000","last_undegraded":"2024-04-24T22:39:38.707809+0000","last_fullsized":"2024-04-24T22:39:38.707809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820143+0000","last_change":"2024-04-24T22:39:36.698983+0000","last_active":"2024-04-24T22:39:38.820143+0000","last_peered":"2024-04-24T22:39:38.820143+0000","last_clean":"2024-04-24T22:39:38.820143+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:39:38.820143+0000","last_undegraded":"2024-04-24T22:39:38.820143+0000","last_fullsized":"2024-04-24T22:39:38.820143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820135+0000","last_change":"2024-04-24T22:39:37.714229+0000","last_active":"2024-04-24T22:39:38.820135+0000","last_peered":"2024-04-24T22:39:38.820135+0000","last_clean":"2024-04-24T22:39:38.820135+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:39:38.820135+0000","last_undegraded":"2024-04-24T22:39:38.820135+0000","last_fullsized":"2024-04-24T22:39:38.820135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819388+0000","last_change":"2024-04-24T22:39:36.705776+0000","last_active":"2024-04-24T22:39:38.819388+0000","last_peered":"2024-04-24T22:39:38.819388+0000","last_clean":"2024-04-24T22:39:38.819388+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:39:38.819388+0000","last_undegraded":"2024-04-24T22:39:38.819388+0000","last_fullsized":"2024-04-24T22:39:38.819388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820936+0000","last_change":"2024-04-24T22:39:37.701436+0000","last_active":"2024-04-24T22:39:38.820936+0000","last_peered":"2024-04-24T22:39:38.820936+0000","last_clean":"2024-04-24T22:39:38.820936+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:39:38.820936+0000","last_undegraded":"2024-04-24T22:39:38.820936+0000","last_fullsized":"2024-04-24T22:39:38.820936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820780+0000","last_change":"2024-04-24T22:39:36.700237+0000","last_active":"2024-04-24T22:39:38.820780+0000","last_peered":"2024-04-24T22:39:38.820780+0000","last_clean":"2024-04-24T22:39:38.820780+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:39:38.820780+0000","last_undegraded":"2024-04-24T22:39:38.820780+0000","last_fullsized":"2024-04-24T22:39:38.820780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705984+0000","last_change":"2024-04-24T22:39:37.703532+0000","last_active":"2024-04-24T22:39:38.705984+0000","last_peered":"2024-04-24T22:39:38.705984+0000","last_clean":"2024-04-24T22:39:38.705984+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:39:38.705984+0000","last_undegraded":"2024-04-24T22:39:38.705984+0000","last_fullsized":"2024-04-24T22:39:38.705984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.724641+0000","last_change":"2024-04-24T22:39:36.717477+0000","last_active":"2024-04-24T22:39:42.724641+0000","last_peered":"2024-04-24T22:39:42.724641+0000","last_clean":"2024-04-24T22:39:42.724641+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:39:42.724641+0000","last_undegraded":"2024-04-24T22:39:42.724641+0000","last_fullsized":"2024-04-24T22:39:42.724641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819989+0000","last_change":"2024-04-24T22:39:37.699255+0000","last_active":"2024-04-24T22:39:38.819989+0000","last_peered":"2024-04-24T22:39:38.819989+0000","last_clean":"2024-04-24T22:39:38.819989+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:39:38.819989+0000","last_undegraded":"2024-04-24T22:39:38.819989+0000","last_fullsized":"2024-04-24T22:39:38.819989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.728917+0000","last_change":"2024-04-24T22:39:36.706567+0000","last_active":"2024-04-24T22:39:42.728917+0000","last_peered":"2024-04-24T22:39:42.728917+0000","last_clean":"2024-04-24T22:39:42.728917+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:39:42.728917+0000","last_undegraded":"2024-04-24T22:39:42.728917+0000","last_fullsized":"2024-04-24T22:39:42.728917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706118+0000","last_change":"2024-04-24T22:39:37.712064+0000","last_active":"2024-04-24T22:39:38.706118+0000","last_peered":"2024-04-24T22:39:38.706118+0000","last_clean":"2024-04-24T22:39:38.706118+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:39:38.706118+0000","last_undegraded":"2024-04-24T22:39:38.706118+0000","last_fullsized":"2024-04-24T22:39:38.706118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.721597+0000","last_change":"2024-04-24T22:39:36.700041+0000","last_active":"2024-04-24T22:39:42.721597+0000","last_peered":"2024-04-24T22:39:42.721597+0000","last_clean":"2024-04-24T22:39:42.721597+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:39:42.721597+0000","last_undegraded":"2024-04-24T22:39:42.721597+0000","last_fullsized":"2024-04-24T22:39:42.721597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:50.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705269+0000","last_change":"2024-04-24T22:39:37.691459+0000","last_active":"2024-04-24T22:39:38.705269+0000","last_peered":"2024-04-24T22:39:38.705269+0000","last_clean":"2024-04-24T22:39:38.705269+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:39:38.705269+0000","last_undegraded":"2024-04-24T22:39:38.705269+0000","last_fullsized":"2024-04-24T22:39:38.705269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820416+0000","last_change":"2024-04-24T22:39:36.706454+0000","last_active":"2024-04-24T22:39:38.820416+0000","last_peered":"2024-04-24T22:39:38.820416+0000","last_clean":"2024-04-24T22:39:38.820416+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:39:38.820416+0000","last_undegraded":"2024-04-24T22:39:38.820416+0000","last_fullsized":"2024-04-24T22:39:38.820416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705898+0000","last_change":"2024-04-24T22:39:37.696724+0000","last_active":"2024-04-24T22:39:38.705898+0000","last_peered":"2024-04-24T22:39:38.705898+0000","last_clean":"2024-04-24T22:39:38.705898+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:39:38.705898+0000","last_undegraded":"2024-04-24T22:39:38.705898+0000","last_fullsized":"2024-04-24T22:39:38.705898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818924+0000","last_change":"2024-04-24T22:39:36.701340+0000","last_active":"2024-04-24T22:39:38.818924+0000","last_peered":"2024-04-24T22:39:38.818924+0000","last_clean":"2024-04-24T22:39:38.818924+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:39:38.818924+0000","last_undegraded":"2024-04-24T22:39:38.818924+0000","last_fullsized":"2024-04-24T22:39:38.818924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819009+0000","last_change":"2024-04-24T22:39:37.697532+0000","last_active":"2024-04-24T22:39:38.819009+0000","last_peered":"2024-04-24T22:39:38.819009+0000","last_clean":"2024-04-24T22:39:38.819009+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:39:38.819009+0000","last_undegraded":"2024-04-24T22:39:38.819009+0000","last_fullsized":"2024-04-24T22:39:38.819009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.727332+0000","last_change":"2024-04-24T22:39:36.724649+0000","last_active":"2024-04-24T22:39:42.727332+0000","last_peered":"2024-04-24T22:39:42.727332+0000","last_clean":"2024-04-24T22:39:42.727332+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:39:42.727332+0000","last_undegraded":"2024-04-24T22:39:42.727332+0000","last_fullsized":"2024-04-24T22:39:42.727332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705839+0000","last_change":"2024-04-24T22:39:37.702890+0000","last_active":"2024-04-24T22:39:38.705839+0000","last_peered":"2024-04-24T22:39:38.705839+0000","last_clean":"2024-04-24T22:39:38.705839+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:39:38.705839+0000","last_undegraded":"2024-04-24T22:39:38.705839+0000","last_fullsized":"2024-04-24T22:39:38.705839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.675479+0000","last_change":"2024-04-24T22:39:36.716044+0000","last_active":"2024-04-24T22:39:39.675479+0000","last_peered":"2024-04-24T22:39:39.675479+0000","last_clean":"2024-04-24T22:39:39.675479+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:39:39.675479+0000","last_undegraded":"2024-04-24T22:39:39.675479+0000","last_fullsized":"2024-04-24T22:39:39.675479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:40.023933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706077+0000","last_change":"2024-04-24T22:39:37.702621+0000","last_active":"2024-04-24T22:39:38.706077+0000","last_peered":"2024-04-24T22:39:38.706077+0000","last_clean":"2024-04-24T22:39:38.706077+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:39:38.706077+0000","last_undegraded":"2024-04-24T22:39:38.706077+0000","last_fullsized":"2024-04-24T22:39:38.706077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.722621+0000","last_change":"2024-04-24T22:39:36.714272+0000","last_active":"2024-04-24T22:39:42.722621+0000","last_peered":"2024-04-24T22:39:42.722621+0000","last_clean":"2024-04-24T22:39:42.722621+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:39:42.722621+0000","last_undegraded":"2024-04-24T22:39:42.722621+0000","last_fullsized":"2024-04-24T22:39:42.722621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706373+0000","last_change":"2024-04-24T22:39:37.703786+0000","last_active":"2024-04-24T22:39:38.706373+0000","last_peered":"2024-04-24T22:39:38.706373+0000","last_clean":"2024-04-24T22:39:38.706373+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:39:38.706373+0000","last_undegraded":"2024-04-24T22:39:38.706373+0000","last_fullsized":"2024-04-24T22:39:38.706373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:55.990656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.687028+0000","last_change":"2024-04-24T22:39:36.715437+0000","last_active":"2024-04-24T22:39:39.687028+0000","last_peered":"2024-04-24T22:39:39.687028+0000","last_clean":"2024-04-24T22:39:39.687028+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:39:39.687028+0000","last_undegraded":"2024-04-24T22:39:39.687028+0000","last_fullsized":"2024-04-24T22:39:39.687028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820735+0000","last_change":"2024-04-24T22:39:37.715801+0000","last_active":"2024-04-24T22:39:38.820735+0000","last_peered":"2024-04-24T22:39:38.820735+0000","last_clean":"2024-04-24T22:39:38.820735+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:39:38.820735+0000","last_undegraded":"2024-04-24T22:39:38.820735+0000","last_fullsized":"2024-04-24T22:39:38.820735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819581+0000","last_change":"2024-04-24T22:39:36.710422+0000","last_active":"2024-04-24T22:39:38.819581+0000","last_peered":"2024-04-24T22:39:38.819581+0000","last_clean":"2024-04-24T22:39:38.819581+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:39:38.819581+0000","last_undegraded":"2024-04-24T22:39:38.819581+0000","last_fullsized":"2024-04-24T22:39:38.819581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706031+0000","last_change":"2024-04-24T22:39:37.706745+0000","last_active":"2024-04-24T22:39:38.706031+0000","last_peered":"2024-04-24T22:39:38.706031+0000","last_clean":"2024-04-24T22:39:38.706031+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:39:38.706031+0000","last_undegraded":"2024-04-24T22:39:38.706031+0000","last_fullsized":"2024-04-24T22:39:38.706031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705671+0000","last_change":"2024-04-24T22:39:32.657965+0000","last_active":"2024-04-24T22:39:38.705671+0000","last_peered":"2024-04-24T22:39:38.705671+0000","last_clean":"2024-04-24T22:39:38.705671+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:39:38.705671+0000","last_undegraded":"2024-04-24T22:39:38.705671+0000","last_fullsized":"2024-04-24T22:39:38.705671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_clean_scrub_stamp":"2024-04-24T22:39:31.635804+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.714153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820290+0000","last_change":"2024-04-24T22:39:36.710381+0000","last_active":"2024-04-24T22:39:38.820290+0000","last_peered":"2024-04-24T22:39:38.820290+0000","last_clean":"2024-04-24T22:39:38.820290+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:39:38.820290+0000","last_undegraded":"2024-04-24T22:39:38.820290+0000","last_fullsized":"2024-04-24T22:39:38.820290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708707+0000","last_change":"2024-04-24T22:39:37.713956+0000","last_active":"2024-04-24T22:39:38.708707+0000","last_peered":"2024-04-24T22:39:38.708707+0000","last_clean":"2024-04-24T22:39:38.708707+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:39:38.708707+0000","last_undegraded":"2024-04-24T22:39:38.708707+0000","last_fullsized":"2024-04-24T22:39:38.708707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705533+0000","last_change":"2024-04-24T22:39:36.715290+0000","last_active":"2024-04-24T22:39:38.705533+0000","last_peered":"2024-04-24T22:39:38.705533+0000","last_clean":"2024-04-24T22:39:38.705533+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:39:38.705533+0000","last_undegraded":"2024-04-24T22:39:38.705533+0000","last_fullsized":"2024-04-24T22:39:38.705533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704538+0000","last_change":"2024-04-24T22:39:37.695419+0000","last_active":"2024-04-24T22:39:38.704538+0000","last_peered":"2024-04-24T22:39:38.704538+0000","last_clean":"2024-04-24T22:39:38.704538+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:39:38.704538+0000","last_undegraded":"2024-04-24T22:39:38.704538+0000","last_fullsized":"2024-04-24T22:39:38.704538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:08.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819608+0000","last_change":"2024-04-24T22:39:36.707548+0000","last_active":"2024-04-24T22:39:38.819608+0000","last_peered":"2024-04-24T22:39:38.819608+0000","last_clean":"2024-04-24T22:39:38.819608+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:39:38.819608+0000","last_undegraded":"2024-04-24T22:39:38.819608+0000","last_fullsized":"2024-04-24T22:39:38.819608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820375+0000","last_change":"2024-04-24T22:39:37.703301+0000","last_active":"2024-04-24T22:39:38.820375+0000","last_peered":"2024-04-24T22:39:38.820375+0000","last_clean":"2024-04-24T22:39:38.820375+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:39:38.820375+0000","last_undegraded":"2024-04-24T22:39:38.820375+0000","last_fullsized":"2024-04-24T22:39:38.820375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820564+0000","last_change":"2024-04-24T22:39:36.713486+0000","last_active":"2024-04-24T22:39:38.820564+0000","last_peered":"2024-04-24T22:39:38.820564+0000","last_clean":"2024-04-24T22:39:38.820564+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:39:38.820564+0000","last_undegraded":"2024-04-24T22:39:38.820564+0000","last_fullsized":"2024-04-24T22:39:38.820564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819526+0000","last_change":"2024-04-24T22:39:37.705705+0000","last_active":"2024-04-24T22:39:38.819526+0000","last_peered":"2024-04-24T22:39:38.819526+0000","last_clean":"2024-04-24T22:39:38.819526+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:39:38.819526+0000","last_undegraded":"2024-04-24T22:39:38.819526+0000","last_fullsized":"2024-04-24T22:39:38.819526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:34.424877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820670+0000","last_change":"2024-04-24T22:39:36.723125+0000","last_active":"2024-04-24T22:39:38.820670+0000","last_peered":"2024-04-24T22:39:38.820670+0000","last_clean":"2024-04-24T22:39:38.820670+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:39:38.820670+0000","last_undegraded":"2024-04-24T22:39:38.820670+0000","last_fullsized":"2024-04-24T22:39:38.820670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820922+0000","last_change":"2024-04-24T22:39:37.713947+0000","last_active":"2024-04-24T22:39:38.820922+0000","last_peered":"2024-04-24T22:39:38.820922+0000","last_clean":"2024-04-24T22:39:38.820922+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:39:38.820922+0000","last_undegraded":"2024-04-24T22:39:38.820922+0000","last_fullsized":"2024-04-24T22:39:38.820922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706225+0000","last_change":"2024-04-24T22:39:36.716541+0000","last_active":"2024-04-24T22:39:38.706225+0000","last_peered":"2024-04-24T22:39:38.706225+0000","last_clean":"2024-04-24T22:39:38.706225+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:39:38.706225+0000","last_undegraded":"2024-04-24T22:39:38.706225+0000","last_fullsized":"2024-04-24T22:39:38.706225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:33.366773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820580+0000","last_change":"2024-04-24T22:39:37.708069+0000","last_active":"2024-04-24T22:39:38.820580+0000","last_peered":"2024-04-24T22:39:38.820580+0000","last_clean":"2024-04-24T22:39:38.820580+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:39:38.820580+0000","last_undegraded":"2024-04-24T22:39:38.820580+0000","last_fullsized":"2024-04-24T22:39:38.820580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.821341+0000","last_change":"2024-04-24T22:39:36.718740+0000","last_active":"2024-04-24T22:39:38.821341+0000","last_peered":"2024-04-24T22:39:38.821341+0000","last_clean":"2024-04-24T22:39:38.821341+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:39:38.821341+0000","last_undegraded":"2024-04-24T22:39:38.821341+0000","last_fullsized":"2024-04-24T22:39:38.821341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.709326+0000","last_change":"2024-04-24T22:39:37.704496+0000","last_active":"2024-04-24T22:39:38.709326+0000","last_peered":"2024-04-24T22:39:38.709326+0000","last_clean":"2024-04-24T22:39:38.709326+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:39:38.709326+0000","last_undegraded":"2024-04-24T22:39:38.709326+0000","last_fullsized":"2024-04-24T22:39:38.709326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.716216+0000","last_change":"2024-04-24T22:39:36.701308+0000","last_active":"2024-04-24T22:39:42.716216+0000","last_peered":"2024-04-24T22:39:42.716216+0000","last_clean":"2024-04-24T22:39:42.716216+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:39:42.716216+0000","last_undegraded":"2024-04-24T22:39:42.716216+0000","last_fullsized":"2024-04-24T22:39:42.716216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705483+0000","last_change":"2024-04-24T22:39:37.700189+0000","last_active":"2024-04-24T22:39:38.705483+0000","last_peered":"2024-04-24T22:39:38.705483+0000","last_clean":"2024-04-24T22:39:38.705483+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:39:38.705483+0000","last_undegraded":"2024-04-24T22:39:38.705483+0000","last_fullsized":"2024-04-24T22:39:38.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705533+0000","last_change":"2024-04-24T22:39:36.716460+0000","last_active":"2024-04-24T22:39:38.705533+0000","last_peered":"2024-04-24T22:39:38.705533+0000","last_clean":"2024-04-24T22:39:38.705533+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:39:38.705533+0000","last_undegraded":"2024-04-24T22:39:38.705533+0000","last_fullsized":"2024-04-24T22:39:38.705533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819020+0000","last_change":"2024-04-24T22:39:37.704143+0000","last_active":"2024-04-24T22:39:38.819020+0000","last_peered":"2024-04-24T22:39:38.819020+0000","last_clean":"2024-04-24T22:39:38.819020+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:39:38.819020+0000","last_undegraded":"2024-04-24T22:39:38.819020+0000","last_fullsized":"2024-04-24T22:39:38.819020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:58.556519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820279+0000","last_change":"2024-04-24T22:39:36.721540+0000","last_active":"2024-04-24T22:39:38.820279+0000","last_peered":"2024-04-24T22:39:38.820279+0000","last_clean":"2024-04-24T22:39:38.820279+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:39:38.820279+0000","last_undegraded":"2024-04-24T22:39:38.820279+0000","last_fullsized":"2024-04-24T22:39:38.820279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708008+0000","last_change":"2024-04-24T22:39:37.714150+0000","last_active":"2024-04-24T22:39:38.708008+0000","last_peered":"2024-04-24T22:39:38.708008+0000","last_clean":"2024-04-24T22:39:38.708008+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:39:38.708008+0000","last_undegraded":"2024-04-24T22:39:38.708008+0000","last_fullsized":"2024-04-24T22:39:38.708008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820479+0000","last_change":"2024-04-24T22:39:36.716667+0000","last_active":"2024-04-24T22:39:38.820479+0000","last_peered":"2024-04-24T22:39:38.820479+0000","last_clean":"2024-04-24T22:39:38.820479+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:39:38.820479+0000","last_undegraded":"2024-04-24T22:39:38.820479+0000","last_fullsized":"2024-04-24T22:39:38.820479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708586+0000","last_change":"2024-04-24T22:39:37.715906+0000","last_active":"2024-04-24T22:39:38.708586+0000","last_peered":"2024-04-24T22:39:38.708586+0000","last_clean":"2024-04-24T22:39:38.708586+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:39:38.708586+0000","last_undegraded":"2024-04-24T22:39:38.708586+0000","last_fullsized":"2024-04-24T22:39:38.708586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.716047+0000","last_change":"2024-04-24T22:39:36.698212+0000","last_active":"2024-04-24T22:39:42.716047+0000","last_peered":"2024-04-24T22:39:42.716047+0000","last_clean":"2024-04-24T22:39:42.716047+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:39:42.716047+0000","last_undegraded":"2024-04-24T22:39:42.716047+0000","last_fullsized":"2024-04-24T22:39:42.716047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818963+0000","last_change":"2024-04-24T22:39:37.696143+0000","last_active":"2024-04-24T22:39:38.818963+0000","last_peered":"2024-04-24T22:39:38.818963+0000","last_clean":"2024-04-24T22:39:38.818963+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:39:38.818963+0000","last_undegraded":"2024-04-24T22:39:38.818963+0000","last_fullsized":"2024-04-24T22:39:38.818963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820834+0000","last_change":"2024-04-24T22:39:37.696113+0000","last_active":"2024-04-24T22:39:38.820834+0000","last_peered":"2024-04-24T22:39:38.820834+0000","last_clean":"2024-04-24T22:39:38.820834+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:39:38.820834+0000","last_undegraded":"2024-04-24T22:39:38.820834+0000","last_fullsized":"2024-04-24T22:39:38.820834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.718503+0000","last_change":"2024-04-24T22:39:36.699149+0000","last_active":"2024-04-24T22:39:42.718503+0000","last_peered":"2024-04-24T22:39:42.718503+0000","last_clean":"2024-04-24T22:39:42.718503+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:39:42.718503+0000","last_undegraded":"2024-04-24T22:39:42.718503+0000","last_fullsized":"2024-04-24T22:39:42.718503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820550+0000","last_change":"2024-04-24T22:39:37.700972+0000","last_active":"2024-04-24T22:39:38.820550+0000","last_peered":"2024-04-24T22:39:38.820550+0000","last_clean":"2024-04-24T22:39:38.820550+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:39:38.820550+0000","last_undegraded":"2024-04-24T22:39:38.820550+0000","last_fullsized":"2024-04-24T22:39:38.820550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.716555+0000","last_change":"2024-04-24T22:39:36.698567+0000","last_active":"2024-04-24T22:39:42.716555+0000","last_peered":"2024-04-24T22:39:42.716555+0000","last_clean":"2024-04-24T22:39:42.716555+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:39:42.716555+0000","last_undegraded":"2024-04-24T22:39:42.716555+0000","last_fullsized":"2024-04-24T22:39:42.716555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706000+0000","last_change":"2024-04-24T22:39:37.706951+0000","last_active":"2024-04-24T22:39:38.706000+0000","last_peered":"2024-04-24T22:39:38.706000+0000","last_clean":"2024-04-24T22:39:38.706000+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:39:38.706000+0000","last_undegraded":"2024-04-24T22:39:38.706000+0000","last_fullsized":"2024-04-24T22:39:38.706000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.683662+0000","last_change":"2024-04-24T22:39:36.703823+0000","last_active":"2024-04-24T22:39:39.683662+0000","last_peered":"2024-04-24T22:39:39.683662+0000","last_clean":"2024-04-24T22:39:39.683662+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:39:39.683662+0000","last_undegraded":"2024-04-24T22:39:39.683662+0000","last_fullsized":"2024-04-24T22:39:39.683662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708601+0000","last_change":"2024-04-24T22:39:37.712078+0000","last_active":"2024-04-24T22:39:38.708601+0000","last_peered":"2024-04-24T22:39:38.708601+0000","last_clean":"2024-04-24T22:39:38.708601+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:39:38.708601+0000","last_undegraded":"2024-04-24T22:39:38.708601+0000","last_fullsized":"2024-04-24T22:39:38.708601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.723764+0000","last_change":"2024-04-24T22:39:36.719643+0000","last_active":"2024-04-24T22:39:42.723764+0000","last_peered":"2024-04-24T22:39:42.723764+0000","last_clean":"2024-04-24T22:39:42.723764+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:39:42.723764+0000","last_undegraded":"2024-04-24T22:39:42.723764+0000","last_fullsized":"2024-04-24T22:39:42.723764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:39:47.939 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:39:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.180+0000 7f499ac00640 1 Processor -- start 2024-04-24T22:39:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.180+0000 7f499ac00640 1 -- start start 2024-04-24T22:39:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f499ac00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4994072c60 0x7f49940c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f499ac00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4994073040 0x7f49941411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f499ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 0x7f4994141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f499ac00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4994158bf0 con 0x7f4994073040 2024-04-24T22:39:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f499ac00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f499415c3d0 con 0x7f4994072c60 2024-04-24T22:39:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f499ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f499415c6b0 con 0x7f499406bb90 2024-04-24T22:39:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f4998e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4994073040 0x7f49941411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f499a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 0x7f4994141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f4998e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4994073040 0x7f49941411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54108/0 (socket says 172.21.15.47:54108) 2024-04-24T22:39:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f4998e00640 1 -- 172.21.15.47:0/2226230323 learned_addr learned my addr 172.21.15.47:0/2226230323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.182+0000 7f4999800640 1 --2- 172.21.15.47:0/2226230323 >> v2:172.21.15.113:3300/0 conn(0x7f4994072c60 0x7f49940c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.183+0000 7f4998e00640 1 -- 172.21.15.47:0/2226230323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 msgr2=0x7f4994141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.183+0000 7f4998e00640 1 --2- 172.21.15.47:0/2226230323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 0x7f4994141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.183+0000 7f4998e00640 1 -- 172.21.15.47:0/2226230323 >> v2:172.21.15.113:3300/0 conn(0x7f4994072c60 msgr2=0x7f49940c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.183+0000 7f4998e00640 1 --2- 172.21.15.47:0/2226230323 >> v2:172.21.15.113:3300/0 conn(0x7f4994072c60 0x7f49940c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.183+0000 7f4998e00640 1 -- 172.21.15.47:0/2226230323 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49941585a0 con 0x7f4994073040 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.183+0000 7f499a200640 1 --2- 172.21.15.47:0/2226230323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 0x7f4994141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.184+0000 7f4998e00640 1 --2- 172.21.15.47:0/2226230323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 0x7f49941411f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f499406b280 tx=0x7f498402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcd4b2616cccacd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.184+0000 7f4993e00640 1 -- 172.21.15.47:0/2226230323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984036070 con 0x7f4994073040 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.184+0000 7f4993e00640 1 -- 172.21.15.47:0/2226230323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4984031600 con 0x7f4994073040 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.184+0000 7f4993e00640 1 -- 172.21.15.47:0/2226230323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49840319d0 con 0x7f4994073040 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.185+0000 7f499ac00640 1 -- 172.21.15.47:0/2226230323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 msgr2=0x7f49941411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.185+0000 7f499ac00640 1 --2- 172.21.15.47:0/2226230323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 0x7f49941411f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f499406b280 tx=0x7f498402a6a0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.185+0000 7f499ac00640 1 -- 172.21.15.47:0/2226230323 shutdown_connections 2024-04-24T22:39:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.185+0000 7f499ac00640 1 --2- 172.21.15.47:0/2226230323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 0x7f4994141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.185+0000 7f499ac00640 1 --2- 172.21.15.47:0/2226230323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 0x7f49941411f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.185+0000 7f499ac00640 1 --2- 172.21.15.47:0/2226230323 >> v2:172.21.15.113:3300/0 conn(0x7f4994072c60 0x7f49940c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.185+0000 7f499ac00640 1 -- 172.21.15.47:0/2226230323 >> 172.21.15.47:0/2226230323 conn(0x7f49940cb900 msgr2=0x7f49940cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.185+0000 7f499ac00640 1 -- 172.21.15.47:0/2226230323 shutdown_connections 2024-04-24T22:39:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.185+0000 7f499ac00640 1 -- 172.21.15.47:0/2226230323 wait complete. 2024-04-24T22:39:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.187+0000 7f499ac00640 1 Processor -- start 2024-04-24T22:39:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.187+0000 7f499ac00640 1 -- start start 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 0x7f49941f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994072c60 0x7f499415ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 0x7f4994153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f49940c32a0 con 0x7f4994073040 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4994158270 con 0x7f4994072c60 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4994159080 con 0x7f499406bb90 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 0x7f4994153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f4999800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 0x7f49941f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f4998e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994072c60 0x7f499415ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f4999800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 0x7f49941f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:59236/0 (socket says 172.21.15.47:59236) 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f4999800640 1 -- 172.21.15.47:0/1959135117 learned_addr learned my addr 172.21.15.47:0/1959135117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 0x7f4994153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54124/0 (socket says 172.21.15.47:54124) 2024-04-24T22:39:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499a200640 1 -- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 msgr2=0x7f49941f1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499a200640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 0x7f49941f1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499a200640 1 -- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994072c60 msgr2=0x7f499415ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499a200640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994072c60 0x7f499415ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f499a200640 1 -- 172.21.15.47:0/1959135117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49941f1740 con 0x7f4994073040 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.188+0000 7f4998e00640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994072c60 0x7f499415ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.189+0000 7f499a200640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 0x7f4994153180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f498c00e7f0 tx=0x7f498c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.189+0000 7f4991600640 1 -- 172.21.15.47:0/1959135117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c017070 con 0x7f4994073040 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.189+0000 7f4991600640 1 -- 172.21.15.47:0/1959135117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f498c00c8c0 con 0x7f4994073040 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.189+0000 7f499ac00640 1 -- 172.21.15.47:0/1959135117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4994154ed0 con 0x7f4994073040 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.189+0000 7f4991600640 1 -- 172.21.15.47:0/1959135117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c00cc70 con 0x7f4994073040 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.189+0000 7f499ac00640 1 -- 172.21.15.47:0/1959135117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4994155490 con 0x7f4994073040 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.191+0000 7f499ac00640 1 -- 172.21.15.47:0/1959135117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4994068d50 con 0x7f4994073040 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.191+0000 7f4991600640 1 -- 172.21.15.47:0/1959135117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f498c01a070 con 0x7f4994073040 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.192+0000 7f4991600640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497806be30 0x7f497806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.192+0000 7f4991600640 1 -- 172.21.15.47:0/1959135117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f498c08afd0 con 0x7f4994073040 2024-04-24T22:39:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.193+0000 7f4999800640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497806be30 0x7f497806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.193+0000 7f4999800640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497806be30 0x7f497806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4988009b60 tx=0x7f4988009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.196+0000 7f4991600640 1 -- 172.21.15.47:0/1959135117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f498c059200 con 0x7f4994073040 2024-04-24T22:39:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.390+0000 7f499ac00640 1 -- 172.21.15.47:0/1959135117 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4994154c70 con 0x7f497806be30 2024-04-24T22:39:48.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.397+0000 7f4991600640 1 -- 172.21.15.47:0/1959135117 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251835 (secure 0 0 0) 0x7f4988011ac0 con 0x7f497806be30 2024-04-24T22:39:48.402 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.402+0000 7f499ac00640 1 -- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497806be30 msgr2=0x7f497806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.402+0000 7f499ac00640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497806be30 0x7f497806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4988009b60 tx=0x7f4988009190 comp rx=0 tx=0).stop 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.403+0000 7f499ac00640 1 -- 172.21.15.47:0/1959135117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 msgr2=0x7f4994153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.403+0000 7f499ac00640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 0x7f4994153180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f498c00e7f0 tx=0x7f498c00ecb0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.403+0000 7f499ac00640 1 -- 172.21.15.47:0/1959135117 shutdown_connections 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.403+0000 7f499ac00640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497806be30 0x7f497806e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.403+0000 7f499ac00640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994073040 0x7f4994153180 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.403+0000 7f499ac00640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994072c60 0x7f499415ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.403+0000 7f499ac00640 1 --2- 172.21.15.47:0/1959135117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499406bb90 0x7f49941f1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.403+0000 7f499ac00640 1 -- 172.21.15.47:0/1959135117 >> 172.21.15.47:0/1959135117 conn(0x7f49940cb900 msgr2=0x7f49940c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.403+0000 7f499ac00640 1 -- 172.21.15.47:0/1959135117 shutdown_connections 2024-04-24T22:39:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.403+0000 7f499ac00640 1 -- 172.21.15.47:0/1959135117 wait complete. 2024-04-24T22:39:48.422 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-24T22:39:46.643666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235688,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739032,"statfs":{"total":773094113280,"available":772852768768,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.969689"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706276+0000","last_change":"2024-04-24T22:39:37.699192+0000","last_active":"2024-04-24T22:39:38.706276+0000","last_peered":"2024-04-24T22:39:38.706276+0000","last_clean":"2024-04-24T22:39:38.706276+0000","last_became_active":"2024-04-24T22:39:37.698920+0000","last_became_peered":"2024-04-24T22:39:37.698920+0000","last_unstale":"2024-04-24T22:39:38.706276+0000","last_undegraded":"2024-04-24T22:39:38.706276+0000","last_fullsized":"2024-04-24T22:39:38.706276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.821236+0000","last_change":"2024-04-24T22:39:37.697245+0000","last_active":"2024-04-24T22:39:38.821236+0000","last_peered":"2024-04-24T22:39:38.821236+0000","last_clean":"2024-04-24T22:39:38.821236+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:39:38.821236+0000","last_undegraded":"2024-04-24T22:39:38.821236+0000","last_fullsized":"2024-04-24T22:39:38.821236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705212+0000","last_change":"2024-04-24T22:39:37.709539+0000","last_active":"2024-04-24T22:39:38.705212+0000","last_peered":"2024-04-24T22:39:38.705212+0000","last_clean":"2024-04-24T22:39:38.705212+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:39:38.705212+0000","last_undegraded":"2024-04-24T22:39:38.705212+0000","last_fullsized":"2024-04-24T22:39:38.705212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820523+0000","last_change":"2024-04-24T22:39:37.719136+0000","last_active":"2024-04-24T22:39:38.820523+0000","last_peered":"2024-04-24T22:39:38.820523+0000","last_clean":"2024-04-24T22:39:38.820523+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:39:38.820523+0000","last_undegraded":"2024-04-24T22:39:38.820523+0000","last_fullsized":"2024-04-24T22:39:38.820523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705940+0000","last_change":"2024-04-24T22:39:37.704671+0000","last_active":"2024-04-24T22:39:38.705940+0000","last_peered":"2024-04-24T22:39:38.705940+0000","last_clean":"2024-04-24T22:39:38.705940+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:39:38.705940+0000","last_undegraded":"2024-04-24T22:39:38.705940+0000","last_fullsized":"2024-04-24T22:39:38.705940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820279+0000","last_change":"2024-04-24T22:39:37.703880+0000","last_active":"2024-04-24T22:39:38.820279+0000","last_peered":"2024-04-24T22:39:38.820279+0000","last_clean":"2024-04-24T22:39:38.820279+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:39:38.820279+0000","last_undegraded":"2024-04-24T22:39:38.820279+0000","last_fullsized":"2024-04-24T22:39:38.820279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705748+0000","last_change":"2024-04-24T22:39:37.703539+0000","last_active":"2024-04-24T22:39:38.705748+0000","last_peered":"2024-04-24T22:39:38.705748+0000","last_clean":"2024-04-24T22:39:38.705748+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:39:38.705748+0000","last_undegraded":"2024-04-24T22:39:38.705748+0000","last_fullsized":"2024-04-24T22:39:38.705748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707950+0000","last_change":"2024-04-24T22:39:37.713543+0000","last_active":"2024-04-24T22:39:38.707950+0000","last_peered":"2024-04-24T22:39:38.707950+0000","last_clean":"2024-04-24T22:39:38.707950+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:39:38.707950+0000","last_undegraded":"2024-04-24T22:39:38.707950+0000","last_fullsized":"2024-04-24T22:39:38.707950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706353+0000","last_change":"2024-04-24T22:39:37.706715+0000","last_active":"2024-04-24T22:39:38.706353+0000","last_peered":"2024-04-24T22:39:38.706353+0000","last_clean":"2024-04-24T22:39:38.706353+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:39:38.706353+0000","last_undegraded":"2024-04-24T22:39:38.706353+0000","last_fullsized":"2024-04-24T22:39:38.706353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819558+0000","last_change":"2024-04-24T22:39:37.701250+0000","last_active":"2024-04-24T22:39:38.819558+0000","last_peered":"2024-04-24T22:39:38.819558+0000","last_clean":"2024-04-24T22:39:38.819558+0000","last_became_active":"2024-04-24T22:39:37.700959+0000","last_became_peered":"2024-04-24T22:39:37.700959+0000","last_unstale":"2024-04-24T22:39:38.819558+0000","last_undegraded":"2024-04-24T22:39:38.819558+0000","last_fullsized":"2024-04-24T22:39:38.819558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819186+0000","last_change":"2024-04-24T22:39:37.701069+0000","last_active":"2024-04-24T22:39:38.819186+0000","last_peered":"2024-04-24T22:39:38.819186+0000","last_clean":"2024-04-24T22:39:38.819186+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:39:38.819186+0000","last_undegraded":"2024-04-24T22:39:38.819186+0000","last_fullsized":"2024-04-24T22:39:38.819186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819923+0000","last_change":"2024-04-24T22:39:37.693508+0000","last_active":"2024-04-24T22:39:38.819923+0000","last_peered":"2024-04-24T22:39:38.819923+0000","last_clean":"2024-04-24T22:39:38.819923+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:39:38.819923+0000","last_undegraded":"2024-04-24T22:39:38.819923+0000","last_fullsized":"2024-04-24T22:39:38.819923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708270+0000","last_change":"2024-04-24T22:39:37.705157+0000","last_active":"2024-04-24T22:39:38.708270+0000","last_peered":"2024-04-24T22:39:38.708270+0000","last_clean":"2024-04-24T22:39:38.708270+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:39:38.708270+0000","last_undegraded":"2024-04-24T22:39:38.708270+0000","last_fullsized":"2024-04-24T22:39:38.708270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704755+0000","last_change":"2024-04-24T22:39:37.698333+0000","last_active":"2024-04-24T22:39:38.704755+0000","last_peered":"2024-04-24T22:39:38.704755+0000","last_clean":"2024-04-24T22:39:38.704755+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:39:38.704755+0000","last_undegraded":"2024-04-24T22:39:38.704755+0000","last_fullsized":"2024-04-24T22:39:38.704755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:57.510197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820219+0000","last_change":"2024-04-24T22:39:37.696874+0000","last_active":"2024-04-24T22:39:38.820219+0000","last_peered":"2024-04-24T22:39:38.820219+0000","last_clean":"2024-04-24T22:39:38.820219+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:39:38.820219+0000","last_undegraded":"2024-04-24T22:39:38.820219+0000","last_fullsized":"2024-04-24T22:39:38.820219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820021+0000","last_change":"2024-04-24T22:39:37.703446+0000","last_active":"2024-04-24T22:39:38.820021+0000","last_peered":"2024-04-24T22:39:38.820021+0000","last_clean":"2024-04-24T22:39:38.820021+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:39:38.820021+0000","last_undegraded":"2024-04-24T22:39:38.820021+0000","last_fullsized":"2024-04-24T22:39:38.820021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820971+0000","last_change":"2024-04-24T22:39:37.701515+0000","last_active":"2024-04-24T22:39:38.820971+0000","last_peered":"2024-04-24T22:39:38.820971+0000","last_clean":"2024-04-24T22:39:38.820971+0000","last_became_active":"2024-04-24T22:39:37.700439+0000","last_became_peered":"2024-04-24T22:39:37.700439+0000","last_unstale":"2024-04-24T22:39:38.820971+0000","last_undegraded":"2024-04-24T22:39:38.820971+0000","last_fullsized":"2024-04-24T22:39:38.820971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819628+0000","last_change":"2024-04-24T22:39:37.703899+0000","last_active":"2024-04-24T22:39:38.819628+0000","last_peered":"2024-04-24T22:39:38.819628+0000","last_clean":"2024-04-24T22:39:38.819628+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:39:38.819628+0000","last_undegraded":"2024-04-24T22:39:38.819628+0000","last_fullsized":"2024-04-24T22:39:38.819628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706244+0000","last_change":"2024-04-24T22:39:37.711019+0000","last_active":"2024-04-24T22:39:38.706244+0000","last_peered":"2024-04-24T22:39:38.706244+0000","last_clean":"2024-04-24T22:39:38.706244+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:39:38.706244+0000","last_undegraded":"2024-04-24T22:39:38.706244+0000","last_fullsized":"2024-04-24T22:39:38.706244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819357+0000","last_change":"2024-04-24T22:39:37.696214+0000","last_active":"2024-04-24T22:39:38.819357+0000","last_peered":"2024-04-24T22:39:38.819357+0000","last_clean":"2024-04-24T22:39:38.819357+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:39:38.819357+0000","last_undegraded":"2024-04-24T22:39:38.819357+0000","last_fullsized":"2024-04-24T22:39:38.819357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705029+0000","last_change":"2024-04-24T22:39:37.700151+0000","last_active":"2024-04-24T22:39:38.705029+0000","last_peered":"2024-04-24T22:39:38.705029+0000","last_clean":"2024-04-24T22:39:38.705029+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:39:38.705029+0000","last_undegraded":"2024-04-24T22:39:38.705029+0000","last_fullsized":"2024-04-24T22:39:38.705029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820267+0000","last_change":"2024-04-24T22:39:37.705083+0000","last_active":"2024-04-24T22:39:38.820267+0000","last_peered":"2024-04-24T22:39:38.820267+0000","last_clean":"2024-04-24T22:39:38.820267+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:39:38.820267+0000","last_undegraded":"2024-04-24T22:39:38.820267+0000","last_fullsized":"2024-04-24T22:39:38.820267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708194+0000","last_change":"2024-04-24T22:39:37.711193+0000","last_active":"2024-04-24T22:39:38.708194+0000","last_peered":"2024-04-24T22:39:38.708194+0000","last_clean":"2024-04-24T22:39:38.708194+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:39:38.708194+0000","last_undegraded":"2024-04-24T22:39:38.708194+0000","last_fullsized":"2024-04-24T22:39:38.708194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820128+0000","last_change":"2024-04-24T22:39:37.713910+0000","last_active":"2024-04-24T22:39:38.820128+0000","last_peered":"2024-04-24T22:39:38.820128+0000","last_clean":"2024-04-24T22:39:38.820128+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:39:38.820128+0000","last_undegraded":"2024-04-24T22:39:38.820128+0000","last_fullsized":"2024-04-24T22:39:38.820128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708266+0000","last_change":"2024-04-24T22:39:37.717191+0000","last_active":"2024-04-24T22:39:38.708266+0000","last_peered":"2024-04-24T22:39:38.708266+0000","last_clean":"2024-04-24T22:39:38.708266+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:39:38.708266+0000","last_undegraded":"2024-04-24T22:39:38.708266+0000","last_fullsized":"2024-04-24T22:39:38.708266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819555+0000","last_change":"2024-04-24T22:39:37.702983+0000","last_active":"2024-04-24T22:39:38.819555+0000","last_peered":"2024-04-24T22:39:38.819555+0000","last_clean":"2024-04-24T22:39:38.819555+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:39:38.819555+0000","last_undegraded":"2024-04-24T22:39:38.819555+0000","last_fullsized":"2024-04-24T22:39:38.819555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820583+0000","last_change":"2024-04-24T22:39:37.714111+0000","last_active":"2024-04-24T22:39:38.820583+0000","last_peered":"2024-04-24T22:39:38.820583+0000","last_clean":"2024-04-24T22:39:38.820583+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:39:38.820583+0000","last_undegraded":"2024-04-24T22:39:38.820583+0000","last_fullsized":"2024-04-24T22:39:38.820583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708923+0000","last_change":"2024-04-24T22:39:37.714877+0000","last_active":"2024-04-24T22:39:38.708923+0000","last_peered":"2024-04-24T22:39:38.708923+0000","last_clean":"2024-04-24T22:39:38.708923+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:39:38.708923+0000","last_undegraded":"2024-04-24T22:39:38.708923+0000","last_fullsized":"2024-04-24T22:39:38.708923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705236+0000","last_change":"2024-04-24T22:39:37.714695+0000","last_active":"2024-04-24T22:39:38.705236+0000","last_peered":"2024-04-24T22:39:38.705236+0000","last_clean":"2024-04-24T22:39:38.705236+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:39:38.705236+0000","last_undegraded":"2024-04-24T22:39:38.705236+0000","last_fullsized":"2024-04-24T22:39:38.705236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820461+0000","last_change":"2024-04-24T22:39:37.719187+0000","last_active":"2024-04-24T22:39:38.820461+0000","last_peered":"2024-04-24T22:39:38.820461+0000","last_clean":"2024-04-24T22:39:38.820461+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:39:38.820461+0000","last_undegraded":"2024-04-24T22:39:38.820461+0000","last_fullsized":"2024-04-24T22:39:38.820461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819723+0000","last_change":"2024-04-24T22:39:37.707283+0000","last_active":"2024-04-24T22:39:38.819723+0000","last_peered":"2024-04-24T22:39:38.819723+0000","last_clean":"2024-04-24T22:39:38.819723+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:39:38.819723+0000","last_undegraded":"2024-04-24T22:39:38.819723+0000","last_fullsized":"2024-04-24T22:39:38.819723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818968+0000","last_change":"2024-04-24T22:39:37.697902+0000","last_active":"2024-04-24T22:39:38.818968+0000","last_peered":"2024-04-24T22:39:38.818968+0000","last_clean":"2024-04-24T22:39:38.818968+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:39:38.818968+0000","last_undegraded":"2024-04-24T22:39:38.818968+0000","last_fullsized":"2024-04-24T22:39:38.818968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819142+0000","last_change":"2024-04-24T22:39:37.705667+0000","last_active":"2024-04-24T22:39:38.819142+0000","last_peered":"2024-04-24T22:39:38.819142+0000","last_clean":"2024-04-24T22:39:38.819142+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:39:38.819142+0000","last_undegraded":"2024-04-24T22:39:38.819142+0000","last_fullsized":"2024-04-24T22:39:38.819142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:50.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.683542+0000","last_change":"2024-04-24T22:39:36.707990+0000","last_active":"2024-04-24T22:39:39.683542+0000","last_peered":"2024-04-24T22:39:39.683542+0000","last_clean":"2024-04-24T22:39:39.683542+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:39:39.683542+0000","last_undegraded":"2024-04-24T22:39:39.683542+0000","last_fullsized":"2024-04-24T22:39:39.683542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706420+0000","last_change":"2024-04-24T22:39:37.700406+0000","last_active":"2024-04-24T22:39:38.706420+0000","last_peered":"2024-04-24T22:39:38.706420+0000","last_clean":"2024-04-24T22:39:38.706420+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:39:38.706420+0000","last_undegraded":"2024-04-24T22:39:38.706420+0000","last_fullsized":"2024-04-24T22:39:38.706420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.675296+0000","last_change":"2024-04-24T22:39:36.701398+0000","last_active":"2024-04-24T22:39:39.675296+0000","last_peered":"2024-04-24T22:39:39.675296+0000","last_clean":"2024-04-24T22:39:39.675296+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:39:39.675296+0000","last_undegraded":"2024-04-24T22:39:39.675296+0000","last_fullsized":"2024-04-24T22:39:39.675296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708466+0000","last_change":"2024-04-24T22:39:36.712294+0000","last_active":"2024-04-24T22:39:38.708466+0000","last_peered":"2024-04-24T22:39:38.708466+0000","last_clean":"2024-04-24T22:39:38.708466+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:39:38.708466+0000","last_undegraded":"2024-04-24T22:39:38.708466+0000","last_fullsized":"2024-04-24T22:39:38.708466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707809+0000","last_change":"2024-04-24T22:39:37.716920+0000","last_active":"2024-04-24T22:39:38.707809+0000","last_peered":"2024-04-24T22:39:38.707809+0000","last_clean":"2024-04-24T22:39:38.707809+0000","last_became_active":"2024-04-24T22:39:37.716641+0000","last_became_peered":"2024-04-24T22:39:37.716641+0000","last_unstale":"2024-04-24T22:39:38.707809+0000","last_undegraded":"2024-04-24T22:39:38.707809+0000","last_fullsized":"2024-04-24T22:39:38.707809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820143+0000","last_change":"2024-04-24T22:39:36.698983+0000","last_active":"2024-04-24T22:39:38.820143+0000","last_peered":"2024-04-24T22:39:38.820143+0000","last_clean":"2024-04-24T22:39:38.820143+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:39:38.820143+0000","last_undegraded":"2024-04-24T22:39:38.820143+0000","last_fullsized":"2024-04-24T22:39:38.820143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820135+0000","last_change":"2024-04-24T22:39:37.714229+0000","last_active":"2024-04-24T22:39:38.820135+0000","last_peered":"2024-04-24T22:39:38.820135+0000","last_clean":"2024-04-24T22:39:38.820135+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:39:38.820135+0000","last_undegraded":"2024-04-24T22:39:38.820135+0000","last_fullsized":"2024-04-24T22:39:38.820135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819388+0000","last_change":"2024-04-24T22:39:36.705776+0000","last_active":"2024-04-24T22:39:38.819388+0000","last_peered":"2024-04-24T22:39:38.819388+0000","last_clean":"2024-04-24T22:39:38.819388+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:39:38.819388+0000","last_undegraded":"2024-04-24T22:39:38.819388+0000","last_fullsized":"2024-04-24T22:39:38.819388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820936+0000","last_change":"2024-04-24T22:39:37.701436+0000","last_active":"2024-04-24T22:39:38.820936+0000","last_peered":"2024-04-24T22:39:38.820936+0000","last_clean":"2024-04-24T22:39:38.820936+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:39:38.820936+0000","last_undegraded":"2024-04-24T22:39:38.820936+0000","last_fullsized":"2024-04-24T22:39:38.820936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820780+0000","last_change":"2024-04-24T22:39:36.700237+0000","last_active":"2024-04-24T22:39:38.820780+0000","last_peered":"2024-04-24T22:39:38.820780+0000","last_clean":"2024-04-24T22:39:38.820780+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:39:38.820780+0000","last_undegraded":"2024-04-24T22:39:38.820780+0000","last_fullsized":"2024-04-24T22:39:38.820780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705984+0000","last_change":"2024-04-24T22:39:37.703532+0000","last_active":"2024-04-24T22:39:38.705984+0000","last_peered":"2024-04-24T22:39:38.705984+0000","last_clean":"2024-04-24T22:39:38.705984+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:39:38.705984+0000","last_undegraded":"2024-04-24T22:39:38.705984+0000","last_fullsized":"2024-04-24T22:39:38.705984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.724641+0000","last_change":"2024-04-24T22:39:36.717477+0000","last_active":"2024-04-24T22:39:42.724641+0000","last_peered":"2024-04-24T22:39:42.724641+0000","last_clean":"2024-04-24T22:39:42.724641+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:39:42.724641+0000","last_undegraded":"2024-04-24T22:39:42.724641+0000","last_fullsized":"2024-04-24T22:39:42.724641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819989+0000","last_change":"2024-04-24T22:39:37.699255+0000","last_active":"2024-04-24T22:39:38.819989+0000","last_peered":"2024-04-24T22:39:38.819989+0000","last_clean":"2024-04-24T22:39:38.819989+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:39:38.819989+0000","last_undegraded":"2024-04-24T22:39:38.819989+0000","last_fullsized":"2024-04-24T22:39:38.819989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.728917+0000","last_change":"2024-04-24T22:39:36.706567+0000","last_active":"2024-04-24T22:39:42.728917+0000","last_peered":"2024-04-24T22:39:42.728917+0000","last_clean":"2024-04-24T22:39:42.728917+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:39:42.728917+0000","last_undegraded":"2024-04-24T22:39:42.728917+0000","last_fullsized":"2024-04-24T22:39:42.728917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706118+0000","last_change":"2024-04-24T22:39:37.712064+0000","last_active":"2024-04-24T22:39:38.706118+0000","last_peered":"2024-04-24T22:39:38.706118+0000","last_clean":"2024-04-24T22:39:38.706118+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:39:38.706118+0000","last_undegraded":"2024-04-24T22:39:38.706118+0000","last_fullsized":"2024-04-24T22:39:38.706118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.721597+0000","last_change":"2024-04-24T22:39:36.700041+0000","last_active":"2024-04-24T22:39:42.721597+0000","last_peered":"2024-04-24T22:39:42.721597+0000","last_clean":"2024-04-24T22:39:42.721597+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:39:42.721597+0000","last_undegraded":"2024-04-24T22:39:42.721597+0000","last_fullsized":"2024-04-24T22:39:42.721597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:50.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705269+0000","last_change":"2024-04-24T22:39:37.691459+0000","last_active":"2024-04-24T22:39:38.705269+0000","last_peered":"2024-04-24T22:39:38.705269+0000","last_clean":"2024-04-24T22:39:38.705269+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:39:38.705269+0000","last_undegraded":"2024-04-24T22:39:38.705269+0000","last_fullsized":"2024-04-24T22:39:38.705269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820416+0000","last_change":"2024-04-24T22:39:36.706454+0000","last_active":"2024-04-24T22:39:38.820416+0000","last_peered":"2024-04-24T22:39:38.820416+0000","last_clean":"2024-04-24T22:39:38.820416+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:39:38.820416+0000","last_undegraded":"2024-04-24T22:39:38.820416+0000","last_fullsized":"2024-04-24T22:39:38.820416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705898+0000","last_change":"2024-04-24T22:39:37.696724+0000","last_active":"2024-04-24T22:39:38.705898+0000","last_peered":"2024-04-24T22:39:38.705898+0000","last_clean":"2024-04-24T22:39:38.705898+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:39:38.705898+0000","last_undegraded":"2024-04-24T22:39:38.705898+0000","last_fullsized":"2024-04-24T22:39:38.705898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818924+0000","last_change":"2024-04-24T22:39:36.701340+0000","last_active":"2024-04-24T22:39:38.818924+0000","last_peered":"2024-04-24T22:39:38.818924+0000","last_clean":"2024-04-24T22:39:38.818924+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:39:38.818924+0000","last_undegraded":"2024-04-24T22:39:38.818924+0000","last_fullsized":"2024-04-24T22:39:38.818924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819009+0000","last_change":"2024-04-24T22:39:37.697532+0000","last_active":"2024-04-24T22:39:38.819009+0000","last_peered":"2024-04-24T22:39:38.819009+0000","last_clean":"2024-04-24T22:39:38.819009+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:39:38.819009+0000","last_undegraded":"2024-04-24T22:39:38.819009+0000","last_fullsized":"2024-04-24T22:39:38.819009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.727332+0000","last_change":"2024-04-24T22:39:36.724649+0000","last_active":"2024-04-24T22:39:42.727332+0000","last_peered":"2024-04-24T22:39:42.727332+0000","last_clean":"2024-04-24T22:39:42.727332+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:39:42.727332+0000","last_undegraded":"2024-04-24T22:39:42.727332+0000","last_fullsized":"2024-04-24T22:39:42.727332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705839+0000","last_change":"2024-04-24T22:39:37.702890+0000","last_active":"2024-04-24T22:39:38.705839+0000","last_peered":"2024-04-24T22:39:38.705839+0000","last_clean":"2024-04-24T22:39:38.705839+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:39:38.705839+0000","last_undegraded":"2024-04-24T22:39:38.705839+0000","last_fullsized":"2024-04-24T22:39:38.705839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.675479+0000","last_change":"2024-04-24T22:39:36.716044+0000","last_active":"2024-04-24T22:39:39.675479+0000","last_peered":"2024-04-24T22:39:39.675479+0000","last_clean":"2024-04-24T22:39:39.675479+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:39:39.675479+0000","last_undegraded":"2024-04-24T22:39:39.675479+0000","last_fullsized":"2024-04-24T22:39:39.675479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:40.023933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706077+0000","last_change":"2024-04-24T22:39:37.702621+0000","last_active":"2024-04-24T22:39:38.706077+0000","last_peered":"2024-04-24T22:39:38.706077+0000","last_clean":"2024-04-24T22:39:38.706077+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:39:38.706077+0000","last_undegraded":"2024-04-24T22:39:38.706077+0000","last_fullsized":"2024-04-24T22:39:38.706077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.722621+0000","last_change":"2024-04-24T22:39:36.714272+0000","last_active":"2024-04-24T22:39:42.722621+0000","last_peered":"2024-04-24T22:39:42.722621+0000","last_clean":"2024-04-24T22:39:42.722621+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:39:42.722621+0000","last_undegraded":"2024-04-24T22:39:42.722621+0000","last_fullsized":"2024-04-24T22:39:42.722621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706373+0000","last_change":"2024-04-24T22:39:37.703786+0000","last_active":"2024-04-24T22:39:38.706373+0000","last_peered":"2024-04-24T22:39:38.706373+0000","last_clean":"2024-04-24T22:39:38.706373+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:39:38.706373+0000","last_undegraded":"2024-04-24T22:39:38.706373+0000","last_fullsized":"2024-04-24T22:39:38.706373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:55.990656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.687028+0000","last_change":"2024-04-24T22:39:36.715437+0000","last_active":"2024-04-24T22:39:39.687028+0000","last_peered":"2024-04-24T22:39:39.687028+0000","last_clean":"2024-04-24T22:39:39.687028+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:39:39.687028+0000","last_undegraded":"2024-04-24T22:39:39.687028+0000","last_fullsized":"2024-04-24T22:39:39.687028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820735+0000","last_change":"2024-04-24T22:39:37.715801+0000","last_active":"2024-04-24T22:39:38.820735+0000","last_peered":"2024-04-24T22:39:38.820735+0000","last_clean":"2024-04-24T22:39:38.820735+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:39:38.820735+0000","last_undegraded":"2024-04-24T22:39:38.820735+0000","last_fullsized":"2024-04-24T22:39:38.820735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819581+0000","last_change":"2024-04-24T22:39:36.710422+0000","last_active":"2024-04-24T22:39:38.819581+0000","last_peered":"2024-04-24T22:39:38.819581+0000","last_clean":"2024-04-24T22:39:38.819581+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:39:38.819581+0000","last_undegraded":"2024-04-24T22:39:38.819581+0000","last_fullsized":"2024-04-24T22:39:38.819581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706031+0000","last_change":"2024-04-24T22:39:37.706745+0000","last_active":"2024-04-24T22:39:38.706031+0000","last_peered":"2024-04-24T22:39:38.706031+0000","last_clean":"2024-04-24T22:39:38.706031+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:39:38.706031+0000","last_undegraded":"2024-04-24T22:39:38.706031+0000","last_fullsized":"2024-04-24T22:39:38.706031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705671+0000","last_change":"2024-04-24T22:39:32.657965+0000","last_active":"2024-04-24T22:39:38.705671+0000","last_peered":"2024-04-24T22:39:38.705671+0000","last_clean":"2024-04-24T22:39:38.705671+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:39:38.705671+0000","last_undegraded":"2024-04-24T22:39:38.705671+0000","last_fullsized":"2024-04-24T22:39:38.705671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_clean_scrub_stamp":"2024-04-24T22:39:31.635804+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.714153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820290+0000","last_change":"2024-04-24T22:39:36.710381+0000","last_active":"2024-04-24T22:39:38.820290+0000","last_peered":"2024-04-24T22:39:38.820290+0000","last_clean":"2024-04-24T22:39:38.820290+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:39:38.820290+0000","last_undegraded":"2024-04-24T22:39:38.820290+0000","last_fullsized":"2024-04-24T22:39:38.820290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708707+0000","last_change":"2024-04-24T22:39:37.713956+0000","last_active":"2024-04-24T22:39:38.708707+0000","last_peered":"2024-04-24T22:39:38.708707+0000","last_clean":"2024-04-24T22:39:38.708707+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:39:38.708707+0000","last_undegraded":"2024-04-24T22:39:38.708707+0000","last_fullsized":"2024-04-24T22:39:38.708707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705533+0000","last_change":"2024-04-24T22:39:36.715290+0000","last_active":"2024-04-24T22:39:38.705533+0000","last_peered":"2024-04-24T22:39:38.705533+0000","last_clean":"2024-04-24T22:39:38.705533+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:39:38.705533+0000","last_undegraded":"2024-04-24T22:39:38.705533+0000","last_fullsized":"2024-04-24T22:39:38.705533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704538+0000","last_change":"2024-04-24T22:39:37.695419+0000","last_active":"2024-04-24T22:39:38.704538+0000","last_peered":"2024-04-24T22:39:38.704538+0000","last_clean":"2024-04-24T22:39:38.704538+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:39:38.704538+0000","last_undegraded":"2024-04-24T22:39:38.704538+0000","last_fullsized":"2024-04-24T22:39:38.704538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:08.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819608+0000","last_change":"2024-04-24T22:39:36.707548+0000","last_active":"2024-04-24T22:39:38.819608+0000","last_peered":"2024-04-24T22:39:38.819608+0000","last_clean":"2024-04-24T22:39:38.819608+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:39:38.819608+0000","last_undegraded":"2024-04-24T22:39:38.819608+0000","last_fullsized":"2024-04-24T22:39:38.819608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820375+0000","last_change":"2024-04-24T22:39:37.703301+0000","last_active":"2024-04-24T22:39:38.820375+0000","last_peered":"2024-04-24T22:39:38.820375+0000","last_clean":"2024-04-24T22:39:38.820375+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:39:38.820375+0000","last_undegraded":"2024-04-24T22:39:38.820375+0000","last_fullsized":"2024-04-24T22:39:38.820375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820564+0000","last_change":"2024-04-24T22:39:36.713486+0000","last_active":"2024-04-24T22:39:38.820564+0000","last_peered":"2024-04-24T22:39:38.820564+0000","last_clean":"2024-04-24T22:39:38.820564+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:39:38.820564+0000","last_undegraded":"2024-04-24T22:39:38.820564+0000","last_fullsized":"2024-04-24T22:39:38.820564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819526+0000","last_change":"2024-04-24T22:39:37.705705+0000","last_active":"2024-04-24T22:39:38.819526+0000","last_peered":"2024-04-24T22:39:38.819526+0000","last_clean":"2024-04-24T22:39:38.819526+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:39:38.819526+0000","last_undegraded":"2024-04-24T22:39:38.819526+0000","last_fullsized":"2024-04-24T22:39:38.819526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:34.424877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820670+0000","last_change":"2024-04-24T22:39:36.723125+0000","last_active":"2024-04-24T22:39:38.820670+0000","last_peered":"2024-04-24T22:39:38.820670+0000","last_clean":"2024-04-24T22:39:38.820670+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:39:38.820670+0000","last_undegraded":"2024-04-24T22:39:38.820670+0000","last_fullsized":"2024-04-24T22:39:38.820670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820922+0000","last_change":"2024-04-24T22:39:37.713947+0000","last_active":"2024-04-24T22:39:38.820922+0000","last_peered":"2024-04-24T22:39:38.820922+0000","last_clean":"2024-04-24T22:39:38.820922+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:39:38.820922+0000","last_undegraded":"2024-04-24T22:39:38.820922+0000","last_fullsized":"2024-04-24T22:39:38.820922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706225+0000","last_change":"2024-04-24T22:39:36.716541+0000","last_active":"2024-04-24T22:39:38.706225+0000","last_peered":"2024-04-24T22:39:38.706225+0000","last_clean":"2024-04-24T22:39:38.706225+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:39:38.706225+0000","last_undegraded":"2024-04-24T22:39:38.706225+0000","last_fullsized":"2024-04-24T22:39:38.706225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:33.366773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820580+0000","last_change":"2024-04-24T22:39:37.708069+0000","last_active":"2024-04-24T22:39:38.820580+0000","last_peered":"2024-04-24T22:39:38.820580+0000","last_clean":"2024-04-24T22:39:38.820580+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:39:38.820580+0000","last_undegraded":"2024-04-24T22:39:38.820580+0000","last_fullsized":"2024-04-24T22:39:38.820580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.821341+0000","last_change":"2024-04-24T22:39:36.718740+0000","last_active":"2024-04-24T22:39:38.821341+0000","last_peered":"2024-04-24T22:39:38.821341+0000","last_clean":"2024-04-24T22:39:38.821341+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:39:38.821341+0000","last_undegraded":"2024-04-24T22:39:38.821341+0000","last_fullsized":"2024-04-24T22:39:38.821341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.709326+0000","last_change":"2024-04-24T22:39:37.704496+0000","last_active":"2024-04-24T22:39:38.709326+0000","last_peered":"2024-04-24T22:39:38.709326+0000","last_clean":"2024-04-24T22:39:38.709326+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:39:38.709326+0000","last_undegraded":"2024-04-24T22:39:38.709326+0000","last_fullsized":"2024-04-24T22:39:38.709326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.716216+0000","last_change":"2024-04-24T22:39:36.701308+0000","last_active":"2024-04-24T22:39:42.716216+0000","last_peered":"2024-04-24T22:39:42.716216+0000","last_clean":"2024-04-24T22:39:42.716216+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:39:42.716216+0000","last_undegraded":"2024-04-24T22:39:42.716216+0000","last_fullsized":"2024-04-24T22:39:42.716216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705483+0000","last_change":"2024-04-24T22:39:37.700189+0000","last_active":"2024-04-24T22:39:38.705483+0000","last_peered":"2024-04-24T22:39:38.705483+0000","last_clean":"2024-04-24T22:39:38.705483+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:39:38.705483+0000","last_undegraded":"2024-04-24T22:39:38.705483+0000","last_fullsized":"2024-04-24T22:39:38.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705533+0000","last_change":"2024-04-24T22:39:36.716460+0000","last_active":"2024-04-24T22:39:38.705533+0000","last_peered":"2024-04-24T22:39:38.705533+0000","last_clean":"2024-04-24T22:39:38.705533+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:39:38.705533+0000","last_undegraded":"2024-04-24T22:39:38.705533+0000","last_fullsized":"2024-04-24T22:39:38.705533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819020+0000","last_change":"2024-04-24T22:39:37.704143+0000","last_active":"2024-04-24T22:39:38.819020+0000","last_peered":"2024-04-24T22:39:38.819020+0000","last_clean":"2024-04-24T22:39:38.819020+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:39:38.819020+0000","last_undegraded":"2024-04-24T22:39:38.819020+0000","last_fullsized":"2024-04-24T22:39:38.819020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:58.556519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820279+0000","last_change":"2024-04-24T22:39:36.721540+0000","last_active":"2024-04-24T22:39:38.820279+0000","last_peered":"2024-04-24T22:39:38.820279+0000","last_clean":"2024-04-24T22:39:38.820279+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:39:38.820279+0000","last_undegraded":"2024-04-24T22:39:38.820279+0000","last_fullsized":"2024-04-24T22:39:38.820279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708008+0000","last_change":"2024-04-24T22:39:37.714150+0000","last_active":"2024-04-24T22:39:38.708008+0000","last_peered":"2024-04-24T22:39:38.708008+0000","last_clean":"2024-04-24T22:39:38.708008+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:39:38.708008+0000","last_undegraded":"2024-04-24T22:39:38.708008+0000","last_fullsized":"2024-04-24T22:39:38.708008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820479+0000","last_change":"2024-04-24T22:39:36.716667+0000","last_active":"2024-04-24T22:39:38.820479+0000","last_peered":"2024-04-24T22:39:38.820479+0000","last_clean":"2024-04-24T22:39:38.820479+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:39:38.820479+0000","last_undegraded":"2024-04-24T22:39:38.820479+0000","last_fullsized":"2024-04-24T22:39:38.820479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708586+0000","last_change":"2024-04-24T22:39:37.715906+0000","last_active":"2024-04-24T22:39:38.708586+0000","last_peered":"2024-04-24T22:39:38.708586+0000","last_clean":"2024-04-24T22:39:38.708586+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:39:38.708586+0000","last_undegraded":"2024-04-24T22:39:38.708586+0000","last_fullsized":"2024-04-24T22:39:38.708586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.716047+0000","last_change":"2024-04-24T22:39:36.698212+0000","last_active":"2024-04-24T22:39:42.716047+0000","last_peered":"2024-04-24T22:39:42.716047+0000","last_clean":"2024-04-24T22:39:42.716047+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:39:42.716047+0000","last_undegraded":"2024-04-24T22:39:42.716047+0000","last_fullsized":"2024-04-24T22:39:42.716047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818963+0000","last_change":"2024-04-24T22:39:37.696143+0000","last_active":"2024-04-24T22:39:38.818963+0000","last_peered":"2024-04-24T22:39:38.818963+0000","last_clean":"2024-04-24T22:39:38.818963+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:39:38.818963+0000","last_undegraded":"2024-04-24T22:39:38.818963+0000","last_fullsized":"2024-04-24T22:39:38.818963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820834+0000","last_change":"2024-04-24T22:39:37.696113+0000","last_active":"2024-04-24T22:39:38.820834+0000","last_peered":"2024-04-24T22:39:38.820834+0000","last_clean":"2024-04-24T22:39:38.820834+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:39:38.820834+0000","last_undegraded":"2024-04-24T22:39:38.820834+0000","last_fullsized":"2024-04-24T22:39:38.820834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.718503+0000","last_change":"2024-04-24T22:39:36.699149+0000","last_active":"2024-04-24T22:39:42.718503+0000","last_peered":"2024-04-24T22:39:42.718503+0000","last_clean":"2024-04-24T22:39:42.718503+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:39:42.718503+0000","last_undegraded":"2024-04-24T22:39:42.718503+0000","last_fullsized":"2024-04-24T22:39:42.718503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820550+0000","last_change":"2024-04-24T22:39:37.700972+0000","last_active":"2024-04-24T22:39:38.820550+0000","last_peered":"2024-04-24T22:39:38.820550+0000","last_clean":"2024-04-24T22:39:38.820550+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:39:38.820550+0000","last_undegraded":"2024-04-24T22:39:38.820550+0000","last_fullsized":"2024-04-24T22:39:38.820550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.716555+0000","last_change":"2024-04-24T22:39:36.698567+0000","last_active":"2024-04-24T22:39:42.716555+0000","last_peered":"2024-04-24T22:39:42.716555+0000","last_clean":"2024-04-24T22:39:42.716555+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:39:42.716555+0000","last_undegraded":"2024-04-24T22:39:42.716555+0000","last_fullsized":"2024-04-24T22:39:42.716555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706000+0000","last_change":"2024-04-24T22:39:37.706951+0000","last_active":"2024-04-24T22:39:38.706000+0000","last_peered":"2024-04-24T22:39:38.706000+0000","last_clean":"2024-04-24T22:39:38.706000+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:39:38.706000+0000","last_undegraded":"2024-04-24T22:39:38.706000+0000","last_fullsized":"2024-04-24T22:39:38.706000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.683662+0000","last_change":"2024-04-24T22:39:36.703823+0000","last_active":"2024-04-24T22:39:39.683662+0000","last_peered":"2024-04-24T22:39:39.683662+0000","last_clean":"2024-04-24T22:39:39.683662+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:39:39.683662+0000","last_undegraded":"2024-04-24T22:39:39.683662+0000","last_fullsized":"2024-04-24T22:39:39.683662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708601+0000","last_change":"2024-04-24T22:39:37.712078+0000","last_active":"2024-04-24T22:39:38.708601+0000","last_peered":"2024-04-24T22:39:38.708601+0000","last_clean":"2024-04-24T22:39:38.708601+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:39:38.708601+0000","last_undegraded":"2024-04-24T22:39:38.708601+0000","last_fullsized":"2024-04-24T22:39:38.708601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.723764+0000","last_change":"2024-04-24T22:39:36.719643+0000","last_active":"2024-04-24T22:39:42.723764+0000","last_peered":"2024-04-24T22:39:42.723764+0000","last_clean":"2024-04-24T22:39:42.723764+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:39:42.723764+0000","last_undegraded":"2024-04-24T22:39:42.723764+0000","last_fullsized":"2024-04-24T22:39:42.723764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:39:48.425 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T22:39:48.426 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T22:39:48.426 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-24T22:39:48.426 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T22:39:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.669+0000 7f279ce00640 1 Processor -- start 2024-04-24T22:39:48.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.670+0000 7f279ce00640 1 -- start start 2024-04-24T22:39:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.671+0000 7f279ce00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2798072c60 0x7f27980c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.671+0000 7f279ce00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798073040 0x7f27981411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.671+0000 7f279ce00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279815c3d0 con 0x7f279806bb90 2024-04-24T22:39:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.671+0000 7f279ce00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2798158bf0 con 0x7f2798072c60 2024-04-24T22:39:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.671+0000 7f279ce00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f279815c6b0 con 0x7f2798073040 2024-04-24T22:39:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.672+0000 7f2797400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2798072c60 0x7f27980c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.672+0000 7f2797e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f279806bb90 0x7f279815b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:51824/0 (socket says 172.21.15.47:51824) 2024-04-24T22:39:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.672+0000 7f2797e00640 1 -- 172.21.15.47:0/2733875858 learned_addr learned my addr 172.21.15.47:0/2733875858 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:48.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.672+0000 7f2796a00640 1 --2- 172.21.15.47:0/2733875858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798073040 0x7f27981411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.672+0000 7f2796000640 1 -- 172.21.15.47:0/2733875858 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1073688560 0 0) 0x7f279815c3d0 con 0x7f279806bb90 2024-04-24T22:39:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.672+0000 7f2796000640 1 -- 172.21.15.47:0/2733875858 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f277c003520 con 0x7f279806bb90 2024-04-24T22:39:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.672+0000 7f2797400640 1 -- 172.21.15.47:0/2733875858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798073040 msgr2=0x7f27981411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.672+0000 7f2797400640 1 --2- 172.21.15.47:0/2733875858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798073040 0x7f27981411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.672+0000 7f2797400640 1 -- 172.21.15.47:0/2733875858 >> v1:172.21.15.113:6789/0 conn(0x7f279806bb90 legacy=0x7f279815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.672+0000 7f2797400640 1 -- 172.21.15.47:0/2733875858 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27981585a0 con 0x7f2798072c60 2024-04-24T22:39:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.673+0000 7f2797400640 1 --2- 172.21.15.47:0/2733875858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2798072c60 0x7f27980c2d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27880097a0 tx=0x7f278802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37b0d3a7156c5fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f2796000640 1 -- 172.21.15.47:0/2733875858 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788037070 con 0x7f2798072c60 2024-04-24T22:39:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f2796000640 1 -- 172.21.15.47:0/2733875858 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2788032440 con 0x7f2798072c60 2024-04-24T22:39:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f2796000640 1 -- 172.21.15.47:0/2733875858 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788032810 con 0x7f2798072c60 2024-04-24T22:39:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f279ce00640 1 -- 172.21.15.47:0/2733875858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2798072c60 msgr2=0x7f27980c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f279ce00640 1 --2- 172.21.15.47:0/2733875858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2798072c60 0x7f27980c2d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27880097a0 tx=0x7f278802a210 comp rx=0 tx=0).stop 2024-04-24T22:39:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f279ce00640 1 -- 172.21.15.47:0/2733875858 shutdown_connections 2024-04-24T22:39:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f279ce00640 1 --2- 172.21.15.47:0/2733875858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798073040 0x7f27981411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f279ce00640 1 --2- 172.21.15.47:0/2733875858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2798072c60 0x7f27980c2d60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f279ce00640 1 -- 172.21.15.47:0/2733875858 >> 172.21.15.47:0/2733875858 conn(0x7f27980cb900 msgr2=0x7f27980cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f279ce00640 1 -- 172.21.15.47:0/2733875858 shutdown_connections 2024-04-24T22:39:48.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.674+0000 7f279ce00640 1 -- 172.21.15.47:0/2733875858 wait complete. 2024-04-24T22:39:48.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.676+0000 7f279ce00640 1 Processor -- start 2024-04-24T22:39:48.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.676+0000 7f279ce00640 1 -- start start 2024-04-24T22:39:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f279ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f279806bb90 0x7f279813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f279ce00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798072c60 0x7f27981ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f279ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2798073040 0x7f27981ee800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f279ce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f27980c32a0 con 0x7f2798073040 2024-04-24T22:39:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f279ce00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2798158270 con 0x7f279806bb90 2024-04-24T22:39:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f279ce00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2798159080 con 0x7f2798072c60 2024-04-24T22:39:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f2797e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2798073040 0x7f27981ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f2797400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f279806bb90 0x7f279813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f2797e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2798073040 0x7f27981ee800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54136/0 (socket says 172.21.15.47:54136) 2024-04-24T22:39:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f2797e00640 1 -- 172.21.15.47:0/2129618690 learned_addr learned my addr 172.21.15.47:0/2129618690 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.677+0000 7f2796a00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798072c60 0x7f27981ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.678+0000 7f2797e00640 1 -- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798072c60 msgr2=0x7f27981ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.678+0000 7f2797e00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798072c60 0x7f27981ee2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.678+0000 7f2797e00640 1 -- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f279806bb90 msgr2=0x7f279813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.678+0000 7f2797e00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f279806bb90 0x7f279813fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.678+0000 7f2797e00640 1 -- 172.21.15.47:0/2129618690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27981f22e0 con 0x7f2798073040 2024-04-24T22:39:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.678+0000 7f2797400640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f279806bb90 0x7f279813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.678+0000 7f2796a00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798072c60 0x7f27981ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.678+0000 7f2797e00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2798073040 0x7f27981ee800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f279000cb40 tx=0x7f279000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:48.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.679+0000 7f278f400640 1 -- 172.21.15.47:0/2129618690 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790017070 con 0x7f2798073040 2024-04-24T22:39:48.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.679+0000 7f278f400640 1 -- 172.21.15.47:0/2129618690 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279000aca0 con 0x7f2798073040 2024-04-24T22:39:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.679+0000 7f278f400640 1 -- 172.21.15.47:0/2129618690 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790012620 con 0x7f2798073040 2024-04-24T22:39:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.679+0000 7f279ce00640 1 -- 172.21.15.47:0/2129618690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279813dda0 con 0x7f2798073040 2024-04-24T22:39:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.679+0000 7f279ce00640 1 -- 172.21.15.47:0/2129618690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279813e360 con 0x7f2798073040 2024-04-24T22:39:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.680+0000 7f279ce00640 1 -- 172.21.15.47:0/2129618690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2798068d50 con 0x7f2798073040 2024-04-24T22:39:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.681+0000 7f278f400640 1 -- 172.21.15.47:0/2129618690 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f279001a070 con 0x7f2798073040 2024-04-24T22:39:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.682+0000 7f278f400640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f277806be30 0x7f277806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.682+0000 7f278f400640 1 -- 172.21.15.47:0/2129618690 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f279008ac20 con 0x7f2798073040 2024-04-24T22:39:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.682+0000 7f2797400640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f277806be30 0x7f277806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.683+0000 7f2797400640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f277806be30 0x7f277806e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2788034000 tx=0x7f278802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:48.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.688+0000 7f278f400640 1 -- 172.21.15.47:0/2129618690 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2790058f80 con 0x7f2798073040 2024-04-24T22:39:48.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.937+0000 7f279ce00640 1 -- 172.21.15.47:0/2129618690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f279806da10 con 0x7f2798073040 2024-04-24T22:39:48.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.937+0000 7f278f400640 1 -- 172.21.15.47:0/2129618690 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f279005d280 con 0x7f2798073040 2024-04-24T22:39:48.939 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:48.939 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-24T22:39:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 -- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f277806be30 msgr2=0x7f277806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f277806be30 0x7f277806e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2788034000 tx=0x7f278802e000 comp rx=0 tx=0).stop 2024-04-24T22:39:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 -- 172.21.15.47:0/2129618690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2798073040 msgr2=0x7f27981ee800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2798073040 0x7f27981ee800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f279000cb40 tx=0x7f279000a060 comp rx=0 tx=0).stop 2024-04-24T22:39:48.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 -- 172.21.15.47:0/2129618690 shutdown_connections 2024-04-24T22:39:48.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f277806be30 0x7f277806e2f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2798073040 0x7f27981ee800 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2798072c60 0x7f27981ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 --2- 172.21.15.47:0/2129618690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f279806bb90 0x7f279813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:48.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 -- 172.21.15.47:0/2129618690 >> 172.21.15.47:0/2129618690 conn(0x7f27980cb900 msgr2=0x7f27981560c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:48.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 -- 172.21.15.47:0/2129618690 shutdown_connections 2024-04-24T22:39:48.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:48.940+0000 7f279ce00640 1 -- 172.21.15.47:0/2129618690 wait complete. 2024-04-24T22:39:50.647 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-04-24T22:39:50.645+0000 7f24c4000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T22:39:50.647 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-04-24T22:39:50.645+0000 7f24c4000640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-24T22:39:51.966 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T22:39:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.165+0000 7f4e56a00640 1 Processor -- start 2024-04-24T22:39:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.166+0000 7f4e56a00640 1 -- start start 2024-04-24T22:39:52.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.167+0000 7f4e56a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e50152960 0x7f4e500c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:52.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.167+0000 7f4e56a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e500c8fa0 0x7f4e5015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:52.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.167+0000 7f4e56a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e5015bf60 0x7f4e50072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:52.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.167+0000 7f4e56a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4e50158f20 con 0x7f4e500c8fa0 2024-04-24T22:39:52.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.167+0000 7f4e56a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4e501593b0 con 0x7f4e5015bf60 2024-04-24T22:39:52.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.167+0000 7f4e56a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4e500c8130 con 0x7f4e50152960 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.167+0000 7f4e4f400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e500c8fa0 0x7f4e5015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.167+0000 7f4e4f400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4e500c8fa0 0x7f4e5015ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54138/0 (socket says 172.21.15.47:54138) 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.167+0000 7f4e4f400640 1 -- 172.21.15.47:0/3792939857 learned_addr learned my addr 172.21.15.47:0/3792939857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.167+0000 7f4e54a00640 1 --2- 172.21.15.47:0/3792939857 >> v2:172.21.15.113:3300/0 conn(0x7f4e5015bf60 0x7f4e50072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.168+0000 7f4e4fe00640 1 --2- 172.21.15.47:0/3792939857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e50152960 0x7f4e500c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.168+0000 7f4e4f400640 1 -- 172.21.15.47:0/3792939857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e50152960 msgr2=0x7f4e500c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.168+0000 7f4e4f400640 1 --2- 172.21.15.47:0/3792939857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e50152960 0x7f4e500c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.168+0000 7f4e4f400640 1 -- 172.21.15.47:0/3792939857 >> v2:172.21.15.113:3300/0 conn(0x7f4e5015bf60 msgr2=0x7f4e50072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.168+0000 7f4e4f400640 1 --2- 172.21.15.47:0/3792939857 >> v2:172.21.15.113:3300/0 conn(0x7f4e5015bf60 0x7f4e50072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.168+0000 7f4e4f400640 1 -- 172.21.15.47:0/3792939857 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e500c9400 con 0x7f4e500c8fa0 2024-04-24T22:39:52.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.168+0000 7f4e4fe00640 1 --2- 172.21.15.47:0/3792939857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e50152960 0x7f4e500c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.169+0000 7f4e4f400640 1 --2- 172.21.15.47:0/3792939857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e500c8fa0 0x7f4e5015ba20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4e5015c3f0 tx=0x7f4e3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d177c44a56144463 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e4ea00640 1 -- 172.21.15.47:0/3792939857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c037070 con 0x7f4e500c8fa0 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e4ea00640 1 -- 172.21.15.47:0/3792939857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3c032440 con 0x7f4e500c8fa0 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e4ea00640 1 -- 172.21.15.47:0/3792939857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c032810 con 0x7f4e500c8fa0 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e56a00640 1 -- 172.21.15.47:0/3792939857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e500c8fa0 msgr2=0x7f4e5015ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e56a00640 1 --2- 172.21.15.47:0/3792939857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e500c8fa0 0x7f4e5015ba20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4e5015c3f0 tx=0x7f4e3c02a270 comp rx=0 tx=0).stop 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e56a00640 1 -- 172.21.15.47:0/3792939857 shutdown_connections 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e56a00640 1 --2- 172.21.15.47:0/3792939857 >> v2:172.21.15.113:3300/0 conn(0x7f4e5015bf60 0x7f4e50072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e56a00640 1 --2- 172.21.15.47:0/3792939857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e500c8fa0 0x7f4e5015ba20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e56a00640 1 --2- 172.21.15.47:0/3792939857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e50152960 0x7f4e500c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e56a00640 1 -- 172.21.15.47:0/3792939857 >> 172.21.15.47:0/3792939857 conn(0x7f4e5006e580 msgr2=0x7f4e5006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e56a00640 1 -- 172.21.15.47:0/3792939857 shutdown_connections 2024-04-24T22:39:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.170+0000 7f4e56a00640 1 -- 172.21.15.47:0/3792939857 wait complete. 2024-04-24T22:39:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.172+0000 7f4e56a00640 1 Processor -- start 2024-04-24T22:39:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.172+0000 7f4e56a00640 1 -- start start 2024-04-24T22:39:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.173+0000 7f4e56a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e500c8fa0 0x7f4e501555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.173+0000 7f4e56a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50152960 0x7f4e5014cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.173+0000 7f4e56a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5015bf60 0x7f4e500c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.173+0000 7f4e56a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4e501588d0 con 0x7f4e50152960 2024-04-24T22:39:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.173+0000 7f4e56a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e501585f0 con 0x7f4e5015bf60 2024-04-24T22:39:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.173+0000 7f4e56a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4e50158bb0 con 0x7f4e500c8fa0 2024-04-24T22:39:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.173+0000 7f4e4f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50152960 0x7f4e5014cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.173+0000 7f4e4f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50152960 0x7f4e5014cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54142/0 (socket says 172.21.15.47:54142) 2024-04-24T22:39:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.173+0000 7f4e4f400640 1 -- 172.21.15.47:0/3538203488 learned_addr learned my addr 172.21.15.47:0/3538203488 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.173+0000 7f4e4fe00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e500c8fa0 0x7f4e501555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e54a00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5015bf60 0x7f4e500c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e4f400640 1 -- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e500c8fa0 msgr2=0x7f4e501555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e4f400640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e500c8fa0 0x7f4e501555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e4f400640 1 -- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5015bf60 msgr2=0x7f4e500c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e4f400640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5015bf60 0x7f4e500c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e4f400640 1 -- 172.21.15.47:0/3538203488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5015ab80 con 0x7f4e50152960 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e54a00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5015bf60 0x7f4e500c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e4f400640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50152960 0x7f4e5014cf00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c02a2b0 tx=0x7f4e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e47e00640 1 -- 172.21.15.47:0/3538203488 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c037040 con 0x7f4e50152960 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e47e00640 1 -- 172.21.15.47:0/3538203488 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3c042070 con 0x7f4e50152960 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e56a00640 1 -- 172.21.15.47:0/3538203488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5015ae40 con 0x7f4e50152960 2024-04-24T22:39:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.174+0000 7f4e47e00640 1 -- 172.21.15.47:0/3538203488 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c030880 con 0x7f4e50152960 2024-04-24T22:39:52.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.175+0000 7f4e56a00640 1 -- 172.21.15.47:0/3538203488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e50159720 con 0x7f4e50152960 2024-04-24T22:39:52.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.176+0000 7f4e56a00640 1 -- 172.21.15.47:0/3538203488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e50073b90 con 0x7f4e50152960 2024-04-24T22:39:52.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.177+0000 7f4e47e00640 1 -- 172.21.15.47:0/3538203488 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e3c030a20 con 0x7f4e50152960 2024-04-24T22:39:52.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.177+0000 7f4e47e00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4e3006be30 0x7f4e3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:52.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.178+0000 7f4e47e00640 1 -- 172.21.15.47:0/3538203488 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4e3c0ab6c0 con 0x7f4e50152960 2024-04-24T22:39:52.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.178+0000 7f4e4fe00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4e3006be30 0x7f4e3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.179+0000 7f4e4fe00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4e3006be30 0x7f4e3006e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009950 tx=0x7f4e40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:52.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.182+0000 7f4e47e00640 1 -- 172.21.15.47:0/3538203488 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4e3c07a060 con 0x7f4e50152960 2024-04-24T22:39:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.426+0000 7f4e56a00640 1 -- 172.21.15.47:0/3538203488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4e5006c200 con 0x7f4e50152960 2024-04-24T22:39:52.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.427+0000 7f4e47e00640 1 -- 172.21.15.47:0/3538203488 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f4e3c079960 con 0x7f4e50152960 2024-04-24T22:39:52.428 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:52.429 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-24T22:39:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 -- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4e3006be30 msgr2=0x7f4e3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4e3006be30 0x7f4e3006e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009950 tx=0x7f4e40008040 comp rx=0 tx=0).stop 2024-04-24T22:39:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 -- 172.21.15.47:0/3538203488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50152960 msgr2=0x7f4e5014cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50152960 0x7f4e5014cf00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c02a2b0 tx=0x7f4e3c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 -- 172.21.15.47:0/3538203488 shutdown_connections 2024-04-24T22:39:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4e3006be30 0x7f4e3006e2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e5015bf60 0x7f4e500c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e50152960 0x7f4e5014cf00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 --2- 172.21.15.47:0/3538203488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e500c8fa0 0x7f4e501555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 -- 172.21.15.47:0/3538203488 >> 172.21.15.47:0/3538203488 conn(0x7f4e5006e580 msgr2=0x7f4e5006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 -- 172.21.15.47:0/3538203488 shutdown_connections 2024-04-24T22:39:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:52.430+0000 7f4e56a00640 1 -- 172.21.15.47:0/3538203488 wait complete. 2024-04-24T22:39:55.451 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T22:39:55.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.638+0000 7fc06fe00640 1 Processor -- start 2024-04-24T22:39:55.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.639+0000 7fc06fe00640 1 -- start start 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.640+0000 7fc06fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0701546a0 0x7fc0700c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.640+0000 7fc06fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc07015e4f0 0x7fc07006c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.640+0000 7fc06fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 0x7fc07015de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.640+0000 7fc06fe00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc0700c5010 con 0x7fc07015e4f0 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.640+0000 7fc06fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc07006b4a0 con 0x7fc0701546a0 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.640+0000 7fc06fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc07006b740 con 0x7fc07015d9f0 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.640+0000 7fc06f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 0x7fc07015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.640+0000 7fc06ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0701546a0 0x7fc0700c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.640+0000 7fc066000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc07015e4f0 0x7fc07006c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.641+0000 7fc06ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0701546a0 0x7fc0700c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37932/0 (socket says 172.21.15.47:37932) 2024-04-24T22:39:55.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.641+0000 7fc06f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 0x7fc07015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:59280/0 (socket says 172.21.15.47:59280) 2024-04-24T22:39:55.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.641+0000 7fc06ea00640 1 -- 172.21.15.47:0/1803566849 learned_addr learned my addr 172.21.15.47:0/1803566849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:55.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.641+0000 7fc06f400640 1 -- 172.21.15.47:0/1803566849 >> v2:172.21.15.113:3300/0 conn(0x7fc0701546a0 msgr2=0x7fc0700c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:55.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.641+0000 7fc06f400640 1 --2- 172.21.15.47:0/1803566849 >> v2:172.21.15.113:3300/0 conn(0x7fc0701546a0 0x7fc0700c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.641+0000 7fc06f400640 1 -- 172.21.15.47:0/1803566849 >> v2:172.21.15.47:3300/0 conn(0x7fc07015e4f0 msgr2=0x7fc07006c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:55.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.641+0000 7fc06f400640 1 --2- 172.21.15.47:0/1803566849 >> v2:172.21.15.47:3300/0 conn(0x7fc07015e4f0 0x7fc07006c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.641+0000 7fc06f400640 1 -- 172.21.15.47:0/1803566849 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0700c5260 con 0x7fc07015d9f0 2024-04-24T22:39:55.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.641+0000 7fc06ea00640 1 --2- 172.21.15.47:0/1803566849 >> v2:172.21.15.113:3300/0 conn(0x7fc0701546a0 0x7fc0700c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:55.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.642+0000 7fc06f400640 1 --2- 172.21.15.47:0/1803566849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 0x7fc07015de00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc0700cc2d0 tx=0x7fc06802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=130118e70e0ffea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:55.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.642+0000 7fc06e000640 1 -- 172.21.15.47:0/1803566849 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068037070 con 0x7fc07015d9f0 2024-04-24T22:39:55.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.642+0000 7fc06e000640 1 -- 172.21.15.47:0/1803566849 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06802ae50 con 0x7fc07015d9f0 2024-04-24T22:39:55.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.642+0000 7fc06e000640 1 -- 172.21.15.47:0/1803566849 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068032660 con 0x7fc07015d9f0 2024-04-24T22:39:55.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.643+0000 7fc06fe00640 1 -- 172.21.15.47:0/1803566849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 msgr2=0x7fc07015de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:55.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.643+0000 7fc06fe00640 1 --2- 172.21.15.47:0/1803566849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 0x7fc07015de00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc0700cc2d0 tx=0x7fc06802a100 comp rx=0 tx=0).stop 2024-04-24T22:39:55.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.643+0000 7fc06fe00640 1 -- 172.21.15.47:0/1803566849 shutdown_connections 2024-04-24T22:39:55.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.643+0000 7fc06fe00640 1 --2- 172.21.15.47:0/1803566849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 0x7fc07015de00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.643+0000 7fc06fe00640 1 --2- 172.21.15.47:0/1803566849 >> v2:172.21.15.47:3300/0 conn(0x7fc07015e4f0 0x7fc07006c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.643+0000 7fc06fe00640 1 --2- 172.21.15.47:0/1803566849 >> v2:172.21.15.113:3300/0 conn(0x7fc0701546a0 0x7fc0700c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.643+0000 7fc06fe00640 1 -- 172.21.15.47:0/1803566849 >> 172.21.15.47:0/1803566849 conn(0x7fc070072d20 msgr2=0x7fc0700cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:55.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.643+0000 7fc06fe00640 1 -- 172.21.15.47:0/1803566849 shutdown_connections 2024-04-24T22:39:55.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.643+0000 7fc06fe00640 1 -- 172.21.15.47:0/1803566849 wait complete. 2024-04-24T22:39:55.647 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-04-24T22:39:55.644+0000 7f24c4000640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-24T22:39:55.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.646+0000 7fc06fe00640 1 Processor -- start 2024-04-24T22:39:55.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.646+0000 7fc06fe00640 1 -- start start 2024-04-24T22:39:55.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.647+0000 7fc06fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0701546a0 0x7fc07015cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.647+0000 7fc06fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 0x7fc0701594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.647+0000 7fc06fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc07015e4f0 0x7fc070159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.647+0000 7fc06fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc0700c6280 con 0x7fc07015e4f0 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.647+0000 7fc06fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc070154a80 con 0x7fc0701546a0 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.647+0000 7fc06fe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc07006b160 con 0x7fc07015d9f0 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc06ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0701546a0 0x7fc07015cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc06ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0701546a0 0x7fc07015cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37946/0 (socket says 172.21.15.47:37946) 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc06ea00640 1 -- 172.21.15.47:0/3184444712 learned_addr learned my addr 172.21.15.47:0/3184444712 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc06f400640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc07015e4f0 0x7fc070159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc066000640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 0x7fc0701594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc06ea00640 1 -- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 msgr2=0x7fc0701594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:55.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc06ea00640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 0x7fc0701594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc06ea00640 1 -- 172.21.15.47:0/3184444712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc07015e4f0 msgr2=0x7fc070159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:55.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc06ea00640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc07015e4f0 0x7fc070159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc06ea00640 1 -- 172.21.15.47:0/3184444712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0701e54a0 con 0x7fc0701546a0 2024-04-24T22:39:55.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.648+0000 7fc06ea00640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0701546a0 0x7fc07015cae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc06000ef30 tx=0x7fc06000b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:55.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.649+0000 7fc067400640 1 -- 172.21.15.47:0/3184444712 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06000bd90 con 0x7fc0701546a0 2024-04-24T22:39:55.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.649+0000 7fc067400640 1 -- 172.21.15.47:0/3184444712 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc060019070 con 0x7fc0701546a0 2024-04-24T22:39:55.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.649+0000 7fc06fe00640 1 -- 172.21.15.47:0/3184444712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc070158a30 con 0x7fc0701546a0 2024-04-24T22:39:55.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.649+0000 7fc067400640 1 -- 172.21.15.47:0/3184444712 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060014600 con 0x7fc0701546a0 2024-04-24T22:39:55.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.649+0000 7fc06fe00640 1 -- 172.21.15.47:0/3184444712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc070158cf0 con 0x7fc0701546a0 2024-04-24T22:39:55.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.650+0000 7fc06fe00640 1 -- 172.21.15.47:0/3184444712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc070068520 con 0x7fc0701546a0 2024-04-24T22:39:55.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.651+0000 7fc067400640 1 -- 172.21.15.47:0/3184444712 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc060005d10 con 0x7fc0701546a0 2024-04-24T22:39:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.652+0000 7fc067400640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc04c06be60 0x7fc04c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:55.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.652+0000 7fc067400640 1 -- 172.21.15.47:0/3184444712 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc06008acf0 con 0x7fc0701546a0 2024-04-24T22:39:55.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.652+0000 7fc066000640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc04c06be60 0x7fc04c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:55.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.653+0000 7fc066000640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc04c06be60 0x7fc04c06e320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc070157df0 tx=0x7fc058009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:55.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.656+0000 7fc067400640 1 -- 172.21.15.47:0/3184444712 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc060059040 con 0x7fc0701546a0 2024-04-24T22:39:55.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.891+0000 7fc06fe00640 1 -- 172.21.15.47:0/3184444712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc07004f5e0 con 0x7fc0701546a0 2024-04-24T22:39:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.892+0000 7fc067400640 1 -- 172.21.15.47:0/3184444712 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7fc06005d340 con 0x7fc0701546a0 2024-04-24T22:39:55.902 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:55.902 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-24T22:39:55.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.894+0000 7fc06fe00640 1 -- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc04c06be60 msgr2=0x7fc04c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:55.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.894+0000 7fc06fe00640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc04c06be60 0x7fc04c06e320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc070157df0 tx=0x7fc058009190 comp rx=0 tx=0).stop 2024-04-24T22:39:55.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.894+0000 7fc06fe00640 1 -- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0701546a0 msgr2=0x7fc07015cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:55.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.894+0000 7fc06fe00640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0701546a0 0x7fc07015cae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc06000ef30 tx=0x7fc06000b480 comp rx=0 tx=0).stop 2024-04-24T22:39:55.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.895+0000 7fc06fe00640 1 -- 172.21.15.47:0/3184444712 shutdown_connections 2024-04-24T22:39:55.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.895+0000 7fc06fe00640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc04c06be60 0x7fc04c06e320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.895+0000 7fc06fe00640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc07015e4f0 0x7fc070159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.895+0000 7fc06fe00640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07015d9f0 0x7fc0701594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.895+0000 7fc06fe00640 1 --2- 172.21.15.47:0/3184444712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0701546a0 0x7fc07015cae0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:55.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.895+0000 7fc06fe00640 1 -- 172.21.15.47:0/3184444712 >> 172.21.15.47:0/3184444712 conn(0x7fc070072d20 msgr2=0x7fc070061ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:55.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.895+0000 7fc06fe00640 1 -- 172.21.15.47:0/3184444712 shutdown_connections 2024-04-24T22:39:55.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:55.895+0000 7fc06fe00640 1 -- 172.21.15.47:0/3184444712 wait complete. 2024-04-24T22:39:58.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T22:39:59.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.110+0000 7f4cbd800640 1 Processor -- start 2024-04-24T22:39:59.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.110+0000 7f4cbd800640 1 -- start start 2024-04-24T22:39:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.112+0000 7f4cbd800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4cb806e1b0 0x7f4cb806e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.112+0000 7f4cbd800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c2850 0x7f4cb8162630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.112+0000 7f4cbd800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cb8161e70 con 0x7f4cb80c6880 2024-04-24T22:39:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.112+0000 7f4cbd800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4cb80c71f0 con 0x7f4cb806e1b0 2024-04-24T22:39:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.112+0000 7f4cbd800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb80c7810 con 0x7f4cb80c2850 2024-04-24T22:39:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.113+0000 7f4cb7e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4cb806e1b0 0x7f4cb806e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.113+0000 7f4cbce00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4cb80c6880 0x7f4cb806b160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44172/0 (socket says 172.21.15.47:44172) 2024-04-24T22:39:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.113+0000 7f4cbce00640 1 -- 172.21.15.47:0/3733466055 learned_addr learned my addr 172.21.15.47:0/3733466055 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:39:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.113+0000 7f4cafe00640 1 --2- 172.21.15.47:0/3733466055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c2850 0x7f4cb8162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.113+0000 7f4cb7e00640 1 -- 172.21.15.47:0/3733466055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c2850 msgr2=0x7f4cb8162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:39:59.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.113+0000 7f4cb7e00640 1 --2- 172.21.15.47:0/3733466055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c2850 0x7f4cb8162630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:59.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.113+0000 7f4cb7e00640 1 -- 172.21.15.47:0/3733466055 >> v1:172.21.15.47:6789/0 conn(0x7f4cb80c6880 legacy=0x7f4cb806b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:59.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.113+0000 7f4cb7e00640 1 -- 172.21.15.47:0/3733466055 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb80c7af0 con 0x7f4cb806e1b0 2024-04-24T22:39:59.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.113+0000 7f4cb7400640 1 -- 172.21.15.47:0/3733466055 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3694657035 0 0) 0x7f4cb8161e70 con 0x7f4cb80c6880 2024-04-24T22:39:59.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.113+0000 7f4cafe00640 1 --2- 172.21.15.47:0/3733466055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c2850 0x7f4cb8162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:39:59.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.114+0000 7f4cb7e00640 1 --2- 172.21.15.47:0/3733466055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 0x7f4cb806e590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80097a0 tx=0x7f4ca802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1e8552df0dba24 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:59.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.115+0000 7f4cb7400640 1 -- 172.21.15.47:0/3733466055 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8037070 con 0x7f4cb806e1b0 2024-04-24T22:39:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.115+0000 7f4cb7400640 1 -- 172.21.15.47:0/3733466055 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca8032440 con 0x7f4cb806e1b0 2024-04-24T22:39:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.115+0000 7f4cb7400640 1 -- 172.21.15.47:0/3733466055 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8032810 con 0x7f4cb806e1b0 2024-04-24T22:39:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.115+0000 7f4cbd800640 1 -- 172.21.15.47:0/3733466055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 msgr2=0x7f4cb806e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.115+0000 7f4cbd800640 1 --2- 172.21.15.47:0/3733466055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 0x7f4cb806e590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80097a0 tx=0x7f4ca802a210 comp rx=0 tx=0).stop 2024-04-24T22:39:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.115+0000 7f4cbd800640 1 -- 172.21.15.47:0/3733466055 shutdown_connections 2024-04-24T22:39:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.115+0000 7f4cbd800640 1 --2- 172.21.15.47:0/3733466055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c2850 0x7f4cb8162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.115+0000 7f4cbd800640 1 --2- 172.21.15.47:0/3733466055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 0x7f4cb806e590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.115+0000 7f4cbd800640 1 -- 172.21.15.47:0/3733466055 >> 172.21.15.47:0/3733466055 conn(0x7f4cb80cb900 msgr2=0x7f4cb80cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.116+0000 7f4cbd800640 1 -- 172.21.15.47:0/3733466055 shutdown_connections 2024-04-24T22:39:59.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.116+0000 7f4cbd800640 1 -- 172.21.15.47:0/3733466055 wait complete. 2024-04-24T22:39:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.117+0000 7f4cbd800640 1 Processor -- start 2024-04-24T22:39:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.118+0000 7f4cbd800640 1 -- start start 2024-04-24T22:39:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cbd800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 0x7f4cb815a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cbd800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb80c2850 0x7f4cb815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cbd800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c6880 0x7f4cb80c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cbd800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb8158e00 con 0x7f4cb806e1b0 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cbd800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb815fcd0 con 0x7f4cb80c2850 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cbd800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb8158f80 con 0x7f4cb80c6880 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cb7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 0x7f4cb815a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cb7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 0x7f4cb815a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43834/0 (socket says 172.21.15.47:43834) 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cb7e00640 1 -- 172.21.15.47:0/2845911433 learned_addr learned my addr 172.21.15.47:0/2845911433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cbce00640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c6880 0x7f4cb80c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cafe00640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb80c2850 0x7f4cb815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cb7e00640 1 -- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c6880 msgr2=0x7f4cb80c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cb7e00640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c6880 0x7f4cb80c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cb7e00640 1 -- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb80c2850 msgr2=0x7f4cb815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cb7e00640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb80c2850 0x7f4cb815b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cb7e00640 1 -- 172.21.15.47:0/2845911433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb815c690 con 0x7f4cb806e1b0 2024-04-24T22:39:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.119+0000 7f4cafe00640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb80c2850 0x7f4cb815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:39:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.120+0000 7f4cb7e00640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 0x7f4cb815a4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80098c0 tx=0x7f4ca80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.120+0000 7f4cb4c00640 1 -- 172.21.15.47:0/2845911433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8037040 con 0x7f4cb806e1b0 2024-04-24T22:39:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.120+0000 7f4cb4c00640 1 -- 172.21.15.47:0/2845911433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca8042070 con 0x7f4cb806e1b0 2024-04-24T22:39:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.120+0000 7f4cb4c00640 1 -- 172.21.15.47:0/2845911433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8030940 con 0x7f4cb806e1b0 2024-04-24T22:39:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.120+0000 7f4cbd800640 1 -- 172.21.15.47:0/2845911433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb815c970 con 0x7f4cb806e1b0 2024-04-24T22:39:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.120+0000 7f4cbd800640 1 -- 172.21.15.47:0/2845911433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb8159430 con 0x7f4cb806e1b0 2024-04-24T22:39:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.121+0000 7f4cbd800640 1 -- 172.21.15.47:0/2845911433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb8068d50 con 0x7f4cb806e1b0 2024-04-24T22:39:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.122+0000 7f4cb4c00640 1 -- 172.21.15.47:0/2845911433 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ca8030ae0 con 0x7f4cb806e1b0 2024-04-24T22:39:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.123+0000 7f4cb4c00640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c9406be30 0x7f4c9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:39:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.123+0000 7f4cb4c00640 1 -- 172.21.15.47:0/2845911433 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4ca80ab710 con 0x7f4cb806e1b0 2024-04-24T22:39:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.124+0000 7f4cafe00640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c9406be30 0x7f4c9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:39:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.124+0000 7f4cafe00640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c9406be30 0x7f4c9406e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8155200 tx=0x7f4ca0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:39:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.133+0000 7f4cb4c00640 1 -- 172.21.15.47:0/2845911433 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ca807a140 con 0x7f4cb806e1b0 2024-04-24T22:39:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.369+0000 7f4cbd800640 1 -- 172.21.15.47:0/2845911433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4cb80c2420 con 0x7f4cb806e1b0 2024-04-24T22:39:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.370+0000 7f4cb4c00640 1 -- 172.21.15.47:0/2845911433 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f4ca8079a40 con 0x7f4cb806e1b0 2024-04-24T22:39:59.372 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:39:59.372 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 -- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c9406be30 msgr2=0x7f4c9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c9406be30 0x7f4c9406e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8155200 tx=0x7f4ca0008040 comp rx=0 tx=0).stop 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 -- 172.21.15.47:0/2845911433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 msgr2=0x7f4cb815a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 0x7f4cb815a4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80098c0 tx=0x7f4ca80046c0 comp rx=0 tx=0).stop 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 -- 172.21.15.47:0/2845911433 shutdown_connections 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c9406be30 0x7f4c9406e2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb80c6880 0x7f4cb80c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb80c2850 0x7f4cb815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 --2- 172.21.15.47:0/2845911433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cb806e1b0 0x7f4cb815a4c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 -- 172.21.15.47:0/2845911433 >> 172.21.15.47:0/2845911433 conn(0x7f4cb80cb900 msgr2=0x7f4cb8154120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 -- 172.21.15.47:0/2845911433 shutdown_connections 2024-04-24T22:39:59.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:39:59.372+0000 7f4cbd800640 1 -- 172.21.15.47:0/2845911433 wait complete. 2024-04-24T22:40:02.394 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T22:40:02.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.587+0000 7f21ca800640 1 Processor -- start 2024-04-24T22:40:02.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.587+0000 7f21ca800640 1 -- start start 2024-04-24T22:40:02.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21ca800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21c40c1a00 0x7f21c4072f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21ca800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21c40734a0 0x7f21c41576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21ca800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c415aa30 con 0x7f21c4151780 2024-04-24T22:40:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21ca800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f21c4073930 con 0x7f21c40c1a00 2024-04-24T22:40:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21ca800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f21c415ad10 con 0x7f21c40734a0 2024-04-24T22:40:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21c3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f21c4151780 0x7f21c40c8590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:45234/0 (socket says 172.21.15.47:45234) 2024-04-24T22:40:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21c3400640 1 -- 172.21.15.47:0/3545272323 learned_addr learned my addr 172.21.15.47:0/3545272323 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21c2a00640 1 --2- 172.21.15.47:0/3545272323 >> v2:172.21.15.113:3300/0 conn(0x7f21c40c1a00 0x7f21c4072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21c3e00640 1 --2- 172.21.15.47:0/3545272323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21c40734a0 0x7f21c41576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:02.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21c2000640 1 -- 172.21.15.47:0/3545272323 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1814433866 0 0) 0x7f21c415aa30 con 0x7f21c4151780 2024-04-24T22:40:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.589+0000 7f21c2000640 1 -- 172.21.15.47:0/3545272323 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21a8003520 con 0x7f21c4151780 2024-04-24T22:40:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.590+0000 7f21c2a00640 1 -- 172.21.15.47:0/3545272323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21c40734a0 msgr2=0x7f21c41576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.590+0000 7f21c2a00640 1 --2- 172.21.15.47:0/3545272323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21c40734a0 0x7f21c41576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.590+0000 7f21c2a00640 1 -- 172.21.15.47:0/3545272323 >> v1:172.21.15.113:6789/0 conn(0x7f21c4151780 legacy=0x7f21c40c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.590+0000 7f21c2a00640 1 -- 172.21.15.47:0/3545272323 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c415aff0 con 0x7f21c40c1a00 2024-04-24T22:40:02.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.591+0000 7f21c2a00640 1 --2- 172.21.15.47:0/3545272323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 0x7f21c4072f60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21b00097a0 tx=0x7f21b002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74c97932db81af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:02.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.591+0000 7f21c2000640 1 -- 172.21.15.47:0/3545272323 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0037070 con 0x7f21c40c1a00 2024-04-24T22:40:02.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.591+0000 7f21c2000640 1 -- 172.21.15.47:0/3545272323 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b0032440 con 0x7f21c40c1a00 2024-04-24T22:40:02.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.591+0000 7f21c2000640 1 -- 172.21.15.47:0/3545272323 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0032810 con 0x7f21c40c1a00 2024-04-24T22:40:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.591+0000 7f21ca800640 1 -- 172.21.15.47:0/3545272323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 msgr2=0x7f21c4072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.591+0000 7f21ca800640 1 --2- 172.21.15.47:0/3545272323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 0x7f21c4072f60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21b00097a0 tx=0x7f21b002a210 comp rx=0 tx=0).stop 2024-04-24T22:40:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.591+0000 7f21ca800640 1 -- 172.21.15.47:0/3545272323 shutdown_connections 2024-04-24T22:40:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.591+0000 7f21ca800640 1 --2- 172.21.15.47:0/3545272323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21c40734a0 0x7f21c41576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.591+0000 7f21ca800640 1 --2- 172.21.15.47:0/3545272323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 0x7f21c4072f60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.591+0000 7f21ca800640 1 -- 172.21.15.47:0/3545272323 >> 172.21.15.47:0/3545272323 conn(0x7f21c406e580 msgr2=0x7f21c406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.592+0000 7f21ca800640 1 -- 172.21.15.47:0/3545272323 shutdown_connections 2024-04-24T22:40:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.592+0000 7f21ca800640 1 -- 172.21.15.47:0/3545272323 wait complete. 2024-04-24T22:40:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.593+0000 7f21ca800640 1 Processor -- start 2024-04-24T22:40:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.593+0000 7f21ca800640 1 -- start start 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.594+0000 7f21ca800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c40734a0 0x7f21c40cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.594+0000 7f21ca800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 0x7f21c4150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.594+0000 7f21ca800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21c4151780 0x7f21c41f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.594+0000 7f21ca800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f21c40c8ca0 con 0x7f21c40734a0 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.594+0000 7f21ca800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f21c415b5b0 con 0x7f21c40c1a00 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.594+0000 7f21ca800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f21c40c8e20 con 0x7f21c4151780 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.595+0000 7f21c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 0x7f21c4150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.595+0000 7f21c3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c40734a0 0x7f21c40cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.595+0000 7f21c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 0x7f21c4150130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:47078/0 (socket says 172.21.15.47:47078) 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.595+0000 7f21c2a00640 1 -- 172.21.15.47:0/4009626045 learned_addr learned my addr 172.21.15.47:0/4009626045 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.595+0000 7f21c3e00640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21c4151780 0x7f21c41f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.595+0000 7f21c2a00640 1 -- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21c4151780 msgr2=0x7f21c41f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:02.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.595+0000 7f21c2a00640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21c4151780 0x7f21c41f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:02.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.595+0000 7f21c2a00640 1 -- 172.21.15.47:0/4009626045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c40734a0 msgr2=0x7f21c40cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:02.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.595+0000 7f21c2a00640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c40734a0 0x7f21c40cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:02.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.595+0000 7f21c2a00640 1 -- 172.21.15.47:0/4009626045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c40cb730 con 0x7f21c40c1a00 2024-04-24T22:40:02.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.596+0000 7f21c2a00640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 0x7f21c4150130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f21b0035040 tx=0x7f21b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:02.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.596+0000 7f21bb400640 1 -- 172.21.15.47:0/4009626045 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0041030 con 0x7f21c40c1a00 2024-04-24T22:40:02.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.596+0000 7f21bb400640 1 -- 172.21.15.47:0/4009626045 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b003d4e0 con 0x7f21c40c1a00 2024-04-24T22:40:02.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.596+0000 7f21bb400640 1 -- 172.21.15.47:0/4009626045 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0037040 con 0x7f21c40c1a00 2024-04-24T22:40:02.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.596+0000 7f21ca800640 1 -- 172.21.15.47:0/4009626045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c40cb9c0 con 0x7f21c40c1a00 2024-04-24T22:40:02.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.596+0000 7f21ca800640 1 -- 172.21.15.47:0/4009626045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c4145410 con 0x7f21c40c1a00 2024-04-24T22:40:02.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.597+0000 7f21ca800640 1 -- 172.21.15.47:0/4009626045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c40c6880 con 0x7f21c40c1a00 2024-04-24T22:40:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.598+0000 7f21bb400640 1 -- 172.21.15.47:0/4009626045 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21b0044070 con 0x7f21c40c1a00 2024-04-24T22:40:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.599+0000 7f21bb400640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f21a406be60 0x7f21a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.599+0000 7f21bb400640 1 -- 172.21.15.47:0/4009626045 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f21b00aafb0 con 0x7f21c40c1a00 2024-04-24T22:40:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.599+0000 7f21c3400640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f21a406be60 0x7f21a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.600+0000 7f21c3400640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f21a406be60 0x7f21a406e320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f21b4005880 tx=0x7f21b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.604+0000 7f21bb400640 1 -- 172.21.15.47:0/4009626045 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f21c40c6880 con 0x7f21c40c1a00 2024-04-24T22:40:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.847+0000 7f21ca800640 1 -- 172.21.15.47:0/4009626045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f21c40b1ce0 con 0x7f21c40c1a00 2024-04-24T22:40:02.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.847+0000 7f21bb400640 1 -- 172.21.15.47:0/4009626045 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f21b00791f0 con 0x7f21c40c1a00 2024-04-24T22:40:02.850 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:02.850 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T22:40:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 -- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f21a406be60 msgr2=0x7f21a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f21a406be60 0x7f21a406e320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f21b4005880 tx=0x7f21b4009040 comp rx=0 tx=0).stop 2024-04-24T22:40:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 -- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 msgr2=0x7f21c4150130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 0x7f21c4150130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f21b0035040 tx=0x7f21b00046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 -- 172.21.15.47:0/4009626045 shutdown_connections 2024-04-24T22:40:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f21a406be60 0x7f21a406e320 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21c4151780 0x7f21c41f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21c40c1a00 0x7f21c4150130 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 --2- 172.21.15.47:0/4009626045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21c40734a0 0x7f21c40cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 -- 172.21.15.47:0/4009626045 >> 172.21.15.47:0/4009626045 conn(0x7f21c406e580 msgr2=0x7f21c4070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 -- 172.21.15.47:0/4009626045 shutdown_connections 2024-04-24T22:40:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:02.850+0000 7f21ca800640 1 -- 172.21.15.47:0/4009626045 wait complete. 2024-04-24T22:40:02.869 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-24T22:40:02.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:40:03.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.050+0000 7f0e12200640 1 Processor -- start 2024-04-24T22:40:03.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.050+0000 7f0e12200640 1 -- start start 2024-04-24T22:40:03.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.051+0000 7f0e12200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e0c152670 0x7f0e0c06bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.051+0000 7f0e12200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e0c06c490 0x7f0e0c158220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.051+0000 7f0e12200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c158760 0x7f0e0c0c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.051+0000 7f0e12200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f0e0c15c750 con 0x7f0e0c06c490 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e12200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0e0c06b930 con 0x7f0e0c152670 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e12200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c15b990 con 0x7f0e0c158760 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e0be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e0c06c490 0x7f0e0c158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e0be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e0c06c490 0x7f0e0c158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43856/0 (socket says 172.21.15.47:43856) 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e0be00640 1 -- 172.21.15.47:0/2282390959 learned_addr learned my addr 172.21.15.47:0/2282390959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e10e00640 1 --2- 172.21.15.47:0/2282390959 >> v2:172.21.15.113:3300/0 conn(0x7f0e0c152670 0x7f0e0c06bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e11800640 1 --2- 172.21.15.47:0/2282390959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c158760 0x7f0e0c0c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e0be00640 1 -- 172.21.15.47:0/2282390959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c158760 msgr2=0x7f0e0c0c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:03.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e0be00640 1 --2- 172.21.15.47:0/2282390959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c158760 0x7f0e0c0c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e0be00640 1 -- 172.21.15.47:0/2282390959 >> v2:172.21.15.113:3300/0 conn(0x7f0e0c152670 msgr2=0x7f0e0c06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e0be00640 1 --2- 172.21.15.47:0/2282390959 >> v2:172.21.15.113:3300/0 conn(0x7f0e0c152670 0x7f0e0c06bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.052+0000 7f0e0be00640 1 -- 172.21.15.47:0/2282390959 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c15bc70 con 0x7f0e0c06c490 2024-04-24T22:40:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.053+0000 7f0e0be00640 1 --2- 172.21.15.47:0/2282390959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c06c490 0x7f0e0c158220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0c6c70 tx=0x7f0df802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9408d803514c5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e0b400640 1 -- 172.21.15.47:0/2282390959 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8037070 con 0x7f0e0c06c490 2024-04-24T22:40:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e0b400640 1 -- 172.21.15.47:0/2282390959 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8032440 con 0x7f0e0c06c490 2024-04-24T22:40:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e0b400640 1 -- 172.21.15.47:0/2282390959 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8032810 con 0x7f0e0c06c490 2024-04-24T22:40:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e12200640 1 -- 172.21.15.47:0/2282390959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c06c490 msgr2=0x7f0e0c158220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e12200640 1 --2- 172.21.15.47:0/2282390959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c06c490 0x7f0e0c158220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0c6c70 tx=0x7f0df802a270 comp rx=0 tx=0).stop 2024-04-24T22:40:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e12200640 1 -- 172.21.15.47:0/2282390959 shutdown_connections 2024-04-24T22:40:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e12200640 1 --2- 172.21.15.47:0/2282390959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c158760 0x7f0e0c0c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e12200640 1 --2- 172.21.15.47:0/2282390959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c06c490 0x7f0e0c158220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e12200640 1 --2- 172.21.15.47:0/2282390959 >> v2:172.21.15.113:3300/0 conn(0x7f0e0c152670 0x7f0e0c06bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e12200640 1 -- 172.21.15.47:0/2282390959 >> 172.21.15.47:0/2282390959 conn(0x7f0e0c0c9410 msgr2=0x7f0e0c0c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e12200640 1 -- 172.21.15.47:0/2282390959 shutdown_connections 2024-04-24T22:40:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.054+0000 7f0e12200640 1 -- 172.21.15.47:0/2282390959 wait complete. 2024-04-24T22:40:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.056+0000 7f0e12200640 1 Processor -- start 2024-04-24T22:40:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.056+0000 7f0e12200640 1 -- start start 2024-04-24T22:40:03.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e12200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c06c490 0x7f0e0c159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e12200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c152670 0x7f0e0c145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e12200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c158760 0x7f0e0c1f0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e12200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c0c9190 con 0x7f0e0c152670 2024-04-24T22:40:03.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e12200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c15c230 con 0x7f0e0c06c490 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e12200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c06b3d0 con 0x7f0e0c158760 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c152670 0x7f0e0c145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c152670 0x7f0e0c145f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43870/0 (socket says 172.21.15.47:43870) 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e0be00640 1 -- 172.21.15.47:0/3615095246 learned_addr learned my addr 172.21.15.47:0/3615095246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e10e00640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c06c490 0x7f0e0c159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.057+0000 7f0e11800640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c158760 0x7f0e0c1f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.058+0000 7f0e0be00640 1 -- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c158760 msgr2=0x7f0e0c1f0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.058+0000 7f0e0be00640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c158760 0x7f0e0c1f0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.058+0000 7f0e0be00640 1 -- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c06c490 msgr2=0x7f0e0c159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.058+0000 7f0e0be00640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c06c490 0x7f0e0c159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.058+0000 7f0e0be00640 1 -- 172.21.15.47:0/3615095246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c1f3770 con 0x7f0e0c152670 2024-04-24T22:40:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.058+0000 7f0e0be00640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c152670 0x7f0e0c145f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0df802a2b0 tx=0x7f0df80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:03.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.058+0000 7f0e08c00640 1 -- 172.21.15.47:0/3615095246 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8041020 con 0x7f0e0c152670 2024-04-24T22:40:03.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.058+0000 7f0e08c00640 1 -- 172.21.15.47:0/3615095246 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df803d620 con 0x7f0e0c152670 2024-04-24T22:40:03.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.058+0000 7f0e08c00640 1 -- 172.21.15.47:0/3615095246 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8037040 con 0x7f0e0c152670 2024-04-24T22:40:03.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.058+0000 7f0e12200640 1 -- 172.21.15.47:0/3615095246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c0c5a60 con 0x7f0e0c152670 2024-04-24T22:40:03.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.059+0000 7f0e12200640 1 -- 172.21.15.47:0/3615095246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c1f3cf0 con 0x7f0e0c152670 2024-04-24T22:40:03.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.060+0000 7f0e12200640 1 -- 172.21.15.47:0/3615095246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c04f5e0 con 0x7f0e0c152670 2024-04-24T22:40:03.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.061+0000 7f0e08c00640 1 -- 172.21.15.47:0/3615095246 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0df8044020 con 0x7f0e0c152670 2024-04-24T22:40:03.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.062+0000 7f0e08c00640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0dec06be80 0x7f0dec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.062+0000 7f0e08c00640 1 -- 172.21.15.47:0/3615095246 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0df80ab420 con 0x7f0e0c152670 2024-04-24T22:40:03.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.062+0000 7f0e10e00640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0dec06be80 0x7f0dec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.063+0000 7f0e10e00640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0dec06be80 0x7f0dec06e340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009b40 tx=0x7f0e00009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.066+0000 7f0e08c00640 1 -- 172.21.15.47:0/3615095246 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0df804b030 con 0x7f0e0c152670 2024-04-24T22:40:03.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.265+0000 7f0e12200640 1 -- 172.21.15.47:0/3615095246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e0c06bd30 con 0x7f0e0c152670 2024-04-24T22:40:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.267+0000 7f0e08c00640 1 -- 172.21.15.47:0/3615095246 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16593 (secure 0 0 0) 0x7f0df80796d0 con 0x7f0e0c152670 2024-04-24T22:40:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:03.270 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:38.692613+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:40:03.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 -- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0dec06be80 msgr2=0x7f0dec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0dec06be80 0x7f0dec06e340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009b40 tx=0x7f0e00009190 comp rx=0 tx=0).stop 2024-04-24T22:40:03.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 -- 172.21.15.47:0/3615095246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c152670 msgr2=0x7f0e0c145f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c152670 0x7f0e0c145f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0df802a2b0 tx=0x7f0df80046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 -- 172.21.15.47:0/3615095246 shutdown_connections 2024-04-24T22:40:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0dec06be80 0x7f0dec06e340 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c158760 0x7f0e0c1f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e0c152670 0x7f0e0c145f30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 --2- 172.21.15.47:0/3615095246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c06c490 0x7f0e0c159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 -- 172.21.15.47:0/3615095246 >> 172.21.15.47:0/3615095246 conn(0x7f0e0c0c9410 msgr2=0x7f0e0c0c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 -- 172.21.15.47:0/3615095246 shutdown_connections 2024-04-24T22:40:03.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.270+0000 7f0e12200640 1 -- 172.21.15.47:0/3615095246 wait complete. 2024-04-24T22:40:03.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:30.711538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:34.803185+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:36.184123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:40:03.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:40:03.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.478+0000 7fd553200640 1 Processor -- start 2024-04-24T22:40:03.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.478+0000 7fd553200640 1 -- start start 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.479+0000 7fd553200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd54c152960 0x7fd54c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd553200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd54c0c8fa0 0x7fd54c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd553200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd54c0c9400 con 0x7fd54c15bf60 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd553200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd54c158f20 con 0x7fd54c152960 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd553200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd54c1593b0 con 0x7fd54c0c8fa0 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd54be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd54c152960 0x7fd54c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd551200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd54c15bf60 0x7fd54c072b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44182/0 (socket says 172.21.15.47:44182) 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd551200640 1 -- 172.21.15.47:0/1502963334 learned_addr learned my addr 172.21.15.47:0/1502963334 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd543e00640 1 --2- 172.21.15.47:0/1502963334 >> v2:172.21.15.113:3300/0 conn(0x7fd54c0c8fa0 0x7fd54c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd54be00640 1 -- 172.21.15.47:0/1502963334 >> v2:172.21.15.113:3300/0 conn(0x7fd54c0c8fa0 msgr2=0x7fd54c15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd54be00640 1 --2- 172.21.15.47:0/1502963334 >> v2:172.21.15.113:3300/0 conn(0x7fd54c0c8fa0 0x7fd54c15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd54be00640 1 -- 172.21.15.47:0/1502963334 >> v1:172.21.15.47:6789/0 conn(0x7fd54c15bf60 legacy=0x7fd54c072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd54be00640 1 -- 172.21.15.47:0/1502963334 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c0c7e50 con 0x7fd54c152960 2024-04-24T22:40:03.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.480+0000 7fd543e00640 1 --2- 172.21.15.47:0/1502963334 >> v2:172.21.15.113:3300/0 conn(0x7fd54c0c8fa0 0x7fd54c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:03.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.482+0000 7fd54be00640 1 --2- 172.21.15.47:0/1502963334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 0x7fd54c0c7750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd54c156080 tx=0x7fd53c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01bb88e97770686 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:03.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.482+0000 7fd54b400640 1 -- 172.21.15.47:0/1502963334 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c037070 con 0x7fd54c152960 2024-04-24T22:40:03.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.482+0000 7fd54b400640 1 -- 172.21.15.47:0/1502963334 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd53c032440 con 0x7fd54c152960 2024-04-24T22:40:03.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.482+0000 7fd54b400640 1 -- 172.21.15.47:0/1502963334 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c032810 con 0x7fd54c152960 2024-04-24T22:40:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.483+0000 7fd553200640 1 -- 172.21.15.47:0/1502963334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 msgr2=0x7fd54c0c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.483+0000 7fd553200640 1 --2- 172.21.15.47:0/1502963334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 0x7fd54c0c7750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd54c156080 tx=0x7fd53c02a250 comp rx=0 tx=0).stop 2024-04-24T22:40:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.483+0000 7fd553200640 1 -- 172.21.15.47:0/1502963334 shutdown_connections 2024-04-24T22:40:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.483+0000 7fd553200640 1 --2- 172.21.15.47:0/1502963334 >> v2:172.21.15.113:3300/0 conn(0x7fd54c0c8fa0 0x7fd54c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.483+0000 7fd553200640 1 --2- 172.21.15.47:0/1502963334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 0x7fd54c0c7750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.483+0000 7fd553200640 1 -- 172.21.15.47:0/1502963334 >> 172.21.15.47:0/1502963334 conn(0x7fd54c06e580 msgr2=0x7fd54c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.483+0000 7fd553200640 1 -- 172.21.15.47:0/1502963334 shutdown_connections 2024-04-24T22:40:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.483+0000 7fd553200640 1 -- 172.21.15.47:0/1502963334 wait complete. 2024-04-24T22:40:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.485+0000 7fd553200640 1 Processor -- start 2024-04-24T22:40:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.485+0000 7fd553200640 1 -- start start 2024-04-24T22:40:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd553200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd54c0c8fa0 0x7fd54c0718c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd553200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 0x7fd54c0c3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd553200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd54c15bf60 0x7fd54c0c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd553200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1588d0 con 0x7fd54c152960 2024-04-24T22:40:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd553200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c0c8130 con 0x7fd54c15bf60 2024-04-24T22:40:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd553200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd54c158bb0 con 0x7fd54c0c8fa0 2024-04-24T22:40:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd543e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 0x7fd54c0c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd543e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 0x7fd54c0c3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43900/0 (socket says 172.21.15.47:43900) 2024-04-24T22:40:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd543e00640 1 -- 172.21.15.47:0/3127519444 learned_addr learned my addr 172.21.15.47:0/3127519444 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd54be00640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd54c0c8fa0 0x7fd54c0718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd551200640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd54c15bf60 0x7fd54c0c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd543e00640 1 -- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd54c0c8fa0 msgr2=0x7fd54c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd543e00640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd54c0c8fa0 0x7fd54c0718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd543e00640 1 -- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd54c15bf60 msgr2=0x7fd54c0c3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd543e00640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd54c15bf60 0x7fd54c0c3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd543e00640 1 -- 172.21.15.47:0/3127519444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c1595f0 con 0x7fd54c152960 2024-04-24T22:40:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd551200640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd54c15bf60 0x7fd54c0c3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:40:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.487+0000 7fd54be00640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd54c0c8fa0 0x7fd54c0718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.488+0000 7fd543e00640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 0x7fd54c0c3670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd53400ec10 tx=0x7fd53400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.488+0000 7fd548c00640 1 -- 172.21.15.47:0/3127519444 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534017070 con 0x7fd54c152960 2024-04-24T22:40:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.488+0000 7fd548c00640 1 -- 172.21.15.47:0/3127519444 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd53400cc30 con 0x7fd54c152960 2024-04-24T22:40:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.488+0000 7fd548c00640 1 -- 172.21.15.47:0/3127519444 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534013470 con 0x7fd54c152960 2024-04-24T22:40:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.488+0000 7fd553200640 1 -- 172.21.15.47:0/3127519444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c1f0500 con 0x7fd54c152960 2024-04-24T22:40:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.488+0000 7fd553200640 1 -- 172.21.15.47:0/3127519444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c1f0ac0 con 0x7fd54c152960 2024-04-24T22:40:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.489+0000 7fd553200640 1 -- 172.21.15.47:0/3127519444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54c06b360 con 0x7fd54c152960 2024-04-24T22:40:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.491+0000 7fd548c00640 1 -- 172.21.15.47:0/3127519444 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd53401a070 con 0x7fd54c152960 2024-04-24T22:40:03.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.492+0000 7fd548c00640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd52806be60 0x7fd52806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.493+0000 7fd548c00640 1 -- 172.21.15.47:0/3127519444 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd53408ad30 con 0x7fd54c152960 2024-04-24T22:40:03.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.493+0000 7fd54be00640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd52806be60 0x7fd52806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.493+0000 7fd54be00640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd52806be60 0x7fd52806e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd53c034000 tx=0x7fd53c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:03.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.497+0000 7fd548c00640 1 -- 172.21.15.47:0/3127519444 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd54c06b360 con 0x7fd54c152960 2024-04-24T22:40:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.696+0000 7fd553200640 1 -- 172.21.15.47:0/3127519444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd54c073b90 con 0x7fd54c152960 2024-04-24T22:40:03.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.697+0000 7fd548c00640 1 -- 172.21.15.47:0/3127519444 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd54c1595f0 con 0x7fd54c152960 2024-04-24T22:40:03.700 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-24T22:40:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.699+0000 7fd553200640 1 -- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd52806be60 msgr2=0x7fd52806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.699+0000 7fd553200640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd52806be60 0x7fd52806e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd53c034000 tx=0x7fd53c02e000 comp rx=0 tx=0).stop 2024-04-24T22:40:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.699+0000 7fd553200640 1 -- 172.21.15.47:0/3127519444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 msgr2=0x7fd54c0c3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.699+0000 7fd553200640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 0x7fd54c0c3670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd53400ec10 tx=0x7fd53400c520 comp rx=0 tx=0).stop 2024-04-24T22:40:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.699+0000 7fd553200640 1 -- 172.21.15.47:0/3127519444 shutdown_connections 2024-04-24T22:40:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.699+0000 7fd553200640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd52806be60 0x7fd52806e320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.699+0000 7fd553200640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd54c15bf60 0x7fd54c0c3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.699+0000 7fd553200640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd54c152960 0x7fd54c0c3670 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.699+0000 7fd553200640 1 --2- 172.21.15.47:0/3127519444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd54c0c8fa0 0x7fd54c0718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.700+0000 7fd553200640 1 -- 172.21.15.47:0/3127519444 >> 172.21.15.47:0/3127519444 conn(0x7fd54c06e580 msgr2=0x7fd54c0ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.700+0000 7fd553200640 1 -- 172.21.15.47:0/3127519444 shutdown_connections 2024-04-24T22:40:03.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.700+0000 7fd553200640 1 -- 172.21.15.47:0/3127519444 wait complete. 2024-04-24T22:40:03.720 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:40:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.914+0000 7f6ea3c00640 1 Processor -- start 2024-04-24T22:40:03.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.914+0000 7f6ea3c00640 1 -- start start 2024-04-24T22:40:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.916+0000 7f6ea3c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6e9c0c1a00 0x7f6e9c072f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.916+0000 7f6ea3c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e9c0738f0 con 0x7f6e9c151780 2024-04-24T22:40:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.916+0000 7f6ea3c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e9c15aa30 con 0x7f6e9c0734a0 2024-04-24T22:40:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.916+0000 7f6ea3c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6e9c15ad10 con 0x7f6e9c0c1a00 2024-04-24T22:40:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.916+0000 7f6ea1200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6e9c151780 0x7f6e9c0c8590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44184/0 (socket says 172.21.15.47:44184) 2024-04-24T22:40:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.916+0000 7f6ea1200640 1 -- 172.21.15.47:0/2764606090 learned_addr learned my addr 172.21.15.47:0/2764606090 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.916+0000 7f6e9be00640 1 --2- 172.21.15.47:0/2764606090 >> v2:172.21.15.113:3300/0 conn(0x7f6e9c0c1a00 0x7f6e9c072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.917+0000 7f6e9b400640 1 -- 172.21.15.47:0/2764606090 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1567286866 0 0) 0x7f6e8c001490 con 0x7f6e9c151780 2024-04-24T22:40:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.917+0000 7f6e9b400640 1 -- 172.21.15.47:0/2764606090 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e80003520 con 0x7f6e9c151780 2024-04-24T22:40:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.917+0000 7f6e9b400640 1 -- 172.21.15.47:0/2764606090 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827940628 0 0) 0x7f6e9c15aa30 con 0x7f6e9c0734a0 2024-04-24T22:40:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.917+0000 7f6e9b400640 1 -- 172.21.15.47:0/2764606090 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e8c001490 con 0x7f6e9c0734a0 2024-04-24T22:40:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.917+0000 7f6e9be00640 1 -- 172.21.15.47:0/2764606090 >> v1:172.21.15.113:6789/0 conn(0x7f6e9c0734a0 legacy=0x7f6e9c1576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.917+0000 7f6e9be00640 1 -- 172.21.15.47:0/2764606090 >> v1:172.21.15.47:6789/0 conn(0x7f6e9c151780 legacy=0x7f6e9c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.917+0000 7f6e9be00640 1 -- 172.21.15.47:0/2764606090 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c15aff0 con 0x7f6e9c0c1a00 2024-04-24T22:40:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.918+0000 7f6e9be00640 1 --2- 172.21.15.47:0/2764606090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c0c1a00 0x7f6e9c072f60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6e840097a0 tx=0x7f6e8402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78e53ae83f6c558 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.918+0000 7f6e9b400640 1 -- 172.21.15.47:0/2764606090 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84037070 con 0x7f6e9c0c1a00 2024-04-24T22:40:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.918+0000 7f6e9b400640 1 -- 172.21.15.47:0/2764606090 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e84032640 con 0x7f6e9c0c1a00 2024-04-24T22:40:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.918+0000 7f6e9b400640 1 -- 172.21.15.47:0/2764606090 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84032a10 con 0x7f6e9c0c1a00 2024-04-24T22:40:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.919+0000 7f6ea3c00640 1 -- 172.21.15.47:0/2764606090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c0c1a00 msgr2=0x7f6e9c072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.919+0000 7f6ea3c00640 1 --2- 172.21.15.47:0/2764606090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c0c1a00 0x7f6e9c072f60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6e840097a0 tx=0x7f6e8402a2c0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.919+0000 7f6ea3c00640 1 -- 172.21.15.47:0/2764606090 shutdown_connections 2024-04-24T22:40:03.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.919+0000 7f6ea3c00640 1 --2- 172.21.15.47:0/2764606090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c0c1a00 0x7f6e9c072f60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.919+0000 7f6ea3c00640 1 -- 172.21.15.47:0/2764606090 >> 172.21.15.47:0/2764606090 conn(0x7f6e9c06e580 msgr2=0x7f6e9c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:03.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.919+0000 7f6ea3c00640 1 -- 172.21.15.47:0/2764606090 shutdown_connections 2024-04-24T22:40:03.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.919+0000 7f6ea3c00640 1 -- 172.21.15.47:0/2764606090 wait complete. 2024-04-24T22:40:03.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.920+0000 7f6ea3c00640 1 Processor -- start 2024-04-24T22:40:03.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.921+0000 7f6ea3c00640 1 -- start start 2024-04-24T22:40:03.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.921+0000 7f6ea3c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c0734a0 0x7f6e9c164270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.921+0000 7f6ea3c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c0c1a00 0x7f6e9c0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.921+0000 7f6ea3c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e9c151780 0x7f6e9c0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.921+0000 7f6ea3c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c0c8ca0 con 0x7f6e9c151780 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.921+0000 7f6ea3c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c15b5b0 con 0x7f6e9c0c1a00 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.921+0000 7f6ea3c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9c0c8e20 con 0x7f6e9c0734a0 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6ea1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e9c151780 0x7f6e9c0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6e9be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c0c1a00 0x7f6e9c0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6ea1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e9c151780 0x7f6e9c0cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43904/0 (socket says 172.21.15.47:43904) 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6ea1200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c0734a0 0x7f6e9c164270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6ea1c00640 1 -- 172.21.15.47:0/3951414950 learned_addr learned my addr 172.21.15.47:0/3951414950 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6ea1c00640 1 -- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c0734a0 msgr2=0x7f6e9c164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6ea1c00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c0734a0 0x7f6e9c164270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6ea1c00640 1 -- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c0c1a00 msgr2=0x7f6e9c0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6ea1c00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c0c1a00 0x7f6e9c0ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6ea1c00640 1 -- 172.21.15.47:0/3951414950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c166020 con 0x7f6e9c151780 2024-04-24T22:40:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6e9be00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c0c1a00 0x7f6e9c0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.922+0000 7f6ea1c00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e9c151780 0x7f6e9c0cad20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400c390 tx=0x7f6e9400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:03.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.923+0000 7f6e98c00640 1 -- 172.21.15.47:0/3951414950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94017070 con 0x7f6e9c151780 2024-04-24T22:40:03.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.923+0000 7f6e98c00640 1 -- 172.21.15.47:0/3951414950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e940052d0 con 0x7f6e9c151780 2024-04-24T22:40:03.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.923+0000 7f6e98c00640 1 -- 172.21.15.47:0/3951414950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e940037f0 con 0x7f6e9c151780 2024-04-24T22:40:03.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.923+0000 7f6ea3c00640 1 -- 172.21.15.47:0/3951414950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9c1f54e0 con 0x7f6e9c151780 2024-04-24T22:40:03.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.923+0000 7f6ea3c00640 1 -- 172.21.15.47:0/3951414950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9c166570 con 0x7f6e9c151780 2024-04-24T22:40:03.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.924+0000 7f6ea3c00640 1 -- 172.21.15.47:0/3951414950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e9c073b90 con 0x7f6e9c151780 2024-04-24T22:40:03.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.925+0000 7f6e98c00640 1 -- 172.21.15.47:0/3951414950 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e9401a070 con 0x7f6e9c151780 2024-04-24T22:40:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.926+0000 7f6e98c00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7806be60 0x7f6e7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.926+0000 7f6e98c00640 1 -- 172.21.15.47:0/3951414950 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6e9408a0f0 con 0x7f6e9c151780 2024-04-24T22:40:03.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.927+0000 7f6ea1200640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7806be60 0x7f6e7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:03.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.927+0000 7f6ea1200640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7806be60 0x7f6e7806e320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c002230 tx=0x7f6e8c001c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:03.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:03.931+0000 7f6e98c00640 1 -- 172.21.15.47:0/3951414950 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e940583b0 con 0x7f6e9c151780 2024-04-24T22:40:04.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.128+0000 7f6ea3c00640 1 -- 172.21.15.47:0/3951414950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6e9c0c16f0 con 0x7f6e9c151780 2024-04-24T22:40:04.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.129+0000 7f6e98c00640 1 -- 172.21.15.47:0/3951414950 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f6e9405c6b0 con 0x7f6e9c151780 2024-04-24T22:40:04.132 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 -- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7806be60 msgr2=0x7f6e7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7806be60 0x7f6e7806e320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c002230 tx=0x7f6e8c001c60 comp rx=0 tx=0).stop 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 -- 172.21.15.47:0/3951414950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e9c151780 msgr2=0x7f6e9c0cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e9c151780 0x7f6e9c0cad20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400c390 tx=0x7f6e9400c850 comp rx=0 tx=0).stop 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 -- 172.21.15.47:0/3951414950 shutdown_connections 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7806be60 0x7f6e7806e320 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e9c151780 0x7f6e9c0cad20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c0c1a00 0x7f6e9c0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 --2- 172.21.15.47:0/3951414950 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c0734a0 0x7f6e9c164270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 -- 172.21.15.47:0/3951414950 >> 172.21.15.47:0/3951414950 conn(0x7f6e9c06e580 msgr2=0x7f6e9c0b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 -- 172.21.15.47:0/3951414950 shutdown_connections 2024-04-24T22:40:04.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.132+0000 7f6ea3c00640 1 -- 172.21.15.47:0/3951414950 wait complete. 2024-04-24T22:40:04.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:40:04.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.344+0000 7f8637e00640 1 Processor -- start 2024-04-24T22:40:04.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.344+0000 7f8637e00640 1 -- start start 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.345+0000 7f8637e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8638154560 0x7f86380cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8638073280 0x7f86380cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86380cc660 0x7f86380ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f863815b2c0 con 0x7f86380cc660 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f863815da90 con 0x7f8638073280 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f863815dd70 con 0x7f8638154560 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86380cc660 0x7f86380ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86380cc660 0x7f86380ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43916/0 (socket says 172.21.15.47:43916) 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637400640 1 -- 172.21.15.47:0/970847091 learned_addr learned my addr 172.21.15.47:0/970847091 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8636a00640 1 --2- 172.21.15.47:0/970847091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8638154560 0x7f86380cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f862e000640 1 --2- 172.21.15.47:0/970847091 >> v2:172.21.15.113:3300/0 conn(0x7f8638073280 0x7f86380cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637400640 1 -- 172.21.15.47:0/970847091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8638154560 msgr2=0x7f86380cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:04.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637400640 1 --2- 172.21.15.47:0/970847091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8638154560 0x7f86380cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637400640 1 -- 172.21.15.47:0/970847091 >> v2:172.21.15.113:3300/0 conn(0x7f8638073280 msgr2=0x7f86380cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637400640 1 --2- 172.21.15.47:0/970847091 >> v2:172.21.15.113:3300/0 conn(0x7f8638073280 0x7f86380cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f8637400640 1 -- 172.21.15.47:0/970847091 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863815e050 con 0x7f86380cc660 2024-04-24T22:40:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.346+0000 7f862e000640 1 --2- 172.21.15.47:0/970847091 >> v2:172.21.15.113:3300/0 conn(0x7f8638073280 0x7f86380cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:40:04.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.347+0000 7f8637400640 1 --2- 172.21.15.47:0/970847091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 0x7f86380ccaf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8638073930 tx=0x7f863002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d825b15bc4bfce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:04.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8636000640 1 -- 172.21.15.47:0/970847091 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630036070 con 0x7f86380cc660 2024-04-24T22:40:04.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8636000640 1 -- 172.21.15.47:0/970847091 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8630031600 con 0x7f86380cc660 2024-04-24T22:40:04.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8636000640 1 -- 172.21.15.47:0/970847091 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300319d0 con 0x7f86380cc660 2024-04-24T22:40:04.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8637e00640 1 -- 172.21.15.47:0/970847091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 msgr2=0x7f86380ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:04.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8637e00640 1 --2- 172.21.15.47:0/970847091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 0x7f86380ccaf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8638073930 tx=0x7f863002a6a0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8637e00640 1 -- 172.21.15.47:0/970847091 shutdown_connections 2024-04-24T22:40:04.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8637e00640 1 --2- 172.21.15.47:0/970847091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 0x7f86380ccaf0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8637e00640 1 --2- 172.21.15.47:0/970847091 >> v2:172.21.15.113:3300/0 conn(0x7f8638073280 0x7f86380cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8637e00640 1 --2- 172.21.15.47:0/970847091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8638154560 0x7f86380cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8637e00640 1 -- 172.21.15.47:0/970847091 >> 172.21.15.47:0/970847091 conn(0x7f86380672c0 msgr2=0x7f8638072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:04.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8637e00640 1 -- 172.21.15.47:0/970847091 shutdown_connections 2024-04-24T22:40:04.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.348+0000 7f8637e00640 1 -- 172.21.15.47:0/970847091 wait complete. 2024-04-24T22:40:04.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.350+0000 7f8637e00640 1 Processor -- start 2024-04-24T22:40:04.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.350+0000 7f8637e00640 1 -- start start 2024-04-24T22:40:04.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f8637e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8638073280 0x7f86380cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f8637e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 0x7f863814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f8637e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8638154560 0x7f86381e58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f8637e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f863815ae80 con 0x7f86380cc660 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f8637e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f86380caad0 con 0x7f8638073280 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f8637e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f863815b000 con 0x7f8638154560 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f862e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 0x7f863814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f8636a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8638073280 0x7f86380cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f862e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 0x7f863814aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43924/0 (socket says 172.21.15.47:43924) 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f862e000640 1 -- 172.21.15.47:0/899132581 learned_addr learned my addr 172.21.15.47:0/899132581 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.351+0000 7f8637400640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8638154560 0x7f86381e58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f862e000640 1 -- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8638154560 msgr2=0x7f86381e58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f862e000640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8638154560 0x7f86381e58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f862e000640 1 -- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8638073280 msgr2=0x7f86380cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f862e000640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8638073280 0x7f86380cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f862e000640 1 -- 172.21.15.47:0/899132581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86381e63a0 con 0x7f86380cc660 2024-04-24T22:40:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f8636a00640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8638073280 0x7f86380cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f862e000640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 0x7f863814aa30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f862000d830 tx=0x7f862000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f862fe00640 1 -- 172.21.15.47:0/899132581 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620018070 con 0x7f86380cc660 2024-04-24T22:40:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f862fe00640 1 -- 172.21.15.47:0/899132581 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862000be10 con 0x7f86380cc660 2024-04-24T22:40:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f862fe00640 1 -- 172.21.15.47:0/899132581 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620004250 con 0x7f86380cc660 2024-04-24T22:40:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f8637e00640 1 -- 172.21.15.47:0/899132581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86381f0870 con 0x7f86380cc660 2024-04-24T22:40:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.352+0000 7f8637e00640 1 -- 172.21.15.47:0/899132581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86381f0e30 con 0x7f86380cc660 2024-04-24T22:40:04.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.353+0000 7f8637e00640 1 -- 172.21.15.47:0/899132581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86380c6080 con 0x7f86380cc660 2024-04-24T22:40:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.355+0000 7f862fe00640 1 -- 172.21.15.47:0/899132581 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86200098b0 con 0x7f86380cc660 2024-04-24T22:40:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.356+0000 7f862fe00640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f861406be30 0x7f861406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.357+0000 7f862fe00640 1 -- 172.21.15.47:0/899132581 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f862008b0f0 con 0x7f86380cc660 2024-04-24T22:40:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.357+0000 7f8636a00640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f861406be30 0x7f861406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.357+0000 7f8636a00640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f861406be30 0x7f861406e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8628009e90 tx=0x7f8628009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:04.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.361+0000 7f862fe00640 1 -- 172.21.15.47:0/899132581 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8620059320 con 0x7f86380cc660 2024-04-24T22:40:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.556+0000 7f8637e00640 1 -- 172.21.15.47:0/899132581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f863804f5e0 con 0x7f86380cc660 2024-04-24T22:40:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.557+0000 7f862fe00640 1 -- 172.21.15.47:0/899132581 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f862005d620 con 0x7f86380cc660 2024-04-24T22:40:04.559 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-24T22:40:04.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 -- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f861406be30 msgr2=0x7f861406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:04.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f861406be30 0x7f861406e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8628009e90 tx=0x7f8628009190 comp rx=0 tx=0).stop 2024-04-24T22:40:04.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 -- 172.21.15.47:0/899132581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 msgr2=0x7f863814aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:04.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 0x7f863814aa30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f862000d830 tx=0x7f862000dcf0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 -- 172.21.15.47:0/899132581 shutdown_connections 2024-04-24T22:40:04.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f861406be30 0x7f861406e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8638154560 0x7f86381e58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86380cc660 0x7f863814aa30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 --2- 172.21.15.47:0/899132581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8638073280 0x7f86380cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 -- 172.21.15.47:0/899132581 >> 172.21.15.47:0/899132581 conn(0x7f86380672c0 msgr2=0x7f86380c4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:04.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 -- 172.21.15.47:0/899132581 shutdown_connections 2024-04-24T22:40:04.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.559+0000 7f8637e00640 1 -- 172.21.15.47:0/899132581 wait complete. 2024-04-24T22:40:04.580 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:40:04.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.774+0000 7fbf9dc00640 1 Processor -- start 2024-04-24T22:40:04.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.774+0000 7fbf9dc00640 1 -- start start 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.776+0000 7fbf9dc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf981514e0 0x7fbf9815a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.776+0000 7fbf9dc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf980c76c0 0x7fbf980caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.776+0000 7fbf9dc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf980cb440 0x7fbf980c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.776+0000 7fbf9dc00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbf981518c0 con 0x7fbf980cb440 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.776+0000 7fbf9dc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbf980c7c10 con 0x7fbf981514e0 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.776+0000 7fbf9dc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9815aec0 con 0x7fbf980c76c0 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf9d200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf980cb440 0x7fbf980c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf97e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf981514e0 0x7fbf9815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf9d200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf980cb440 0x7fbf980c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43936/0 (socket says 172.21.15.47:43936) 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf9d200640 1 -- 172.21.15.47:0/228717177 learned_addr learned my addr 172.21.15.47:0/228717177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf97400640 1 --2- 172.21.15.47:0/228717177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf980c76c0 0x7fbf980caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf9d200640 1 -- 172.21.15.47:0/228717177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf980c76c0 msgr2=0x7fbf980caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:04.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf9d200640 1 --2- 172.21.15.47:0/228717177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf980c76c0 0x7fbf980caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf9d200640 1 -- 172.21.15.47:0/228717177 >> v2:172.21.15.113:3300/0 conn(0x7fbf981514e0 msgr2=0x7fbf9815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:04.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf9d200640 1 --2- 172.21.15.47:0/228717177 >> v2:172.21.15.113:3300/0 conn(0x7fbf981514e0 0x7fbf9815a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf9d200640 1 -- 172.21.15.47:0/228717177 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9815b1a0 con 0x7fbf980cb440 2024-04-24T22:40:04.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.777+0000 7fbf97e00640 1 --2- 172.21.15.47:0/228717177 >> v2:172.21.15.113:3300/0 conn(0x7fbf981514e0 0x7fbf9815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:04.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.778+0000 7fbf9d200640 1 --2- 172.21.15.47:0/228717177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 0x7fbf980c8660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbf980ca270 tx=0x7fbf9002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42073cb3ca28b0b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:04.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.779+0000 7fbf96a00640 1 -- 172.21.15.47:0/228717177 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90036070 con 0x7fbf980cb440 2024-04-24T22:40:04.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.779+0000 7fbf96a00640 1 -- 172.21.15.47:0/228717177 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf90031600 con 0x7fbf980cb440 2024-04-24T22:40:04.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.779+0000 7fbf96a00640 1 -- 172.21.15.47:0/228717177 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf900319d0 con 0x7fbf980cb440 2024-04-24T22:40:04.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.779+0000 7fbf9dc00640 1 -- 172.21.15.47:0/228717177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 msgr2=0x7fbf980c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:04.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.779+0000 7fbf9dc00640 1 --2- 172.21.15.47:0/228717177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 0x7fbf980c8660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbf980ca270 tx=0x7fbf9002a730 comp rx=0 tx=0).stop 2024-04-24T22:40:04.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.779+0000 7fbf9dc00640 1 -- 172.21.15.47:0/228717177 shutdown_connections 2024-04-24T22:40:04.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.779+0000 7fbf9dc00640 1 --2- 172.21.15.47:0/228717177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 0x7fbf980c8660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.779+0000 7fbf9dc00640 1 --2- 172.21.15.47:0/228717177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf980c76c0 0x7fbf980caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.779+0000 7fbf9dc00640 1 --2- 172.21.15.47:0/228717177 >> v2:172.21.15.113:3300/0 conn(0x7fbf981514e0 0x7fbf9815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.779+0000 7fbf9dc00640 1 -- 172.21.15.47:0/228717177 >> 172.21.15.47:0/228717177 conn(0x7fbf980c9410 msgr2=0x7fbf980c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:04.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.780+0000 7fbf9dc00640 1 -- 172.21.15.47:0/228717177 shutdown_connections 2024-04-24T22:40:04.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.780+0000 7fbf9dc00640 1 -- 172.21.15.47:0/228717177 wait complete. 2024-04-24T22:40:04.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.781+0000 7fbf9dc00640 1 Processor -- start 2024-04-24T22:40:04.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.781+0000 7fbf9dc00640 1 -- start start 2024-04-24T22:40:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.782+0000 7fbf9dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf980c76c0 0x7fbf981f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.782+0000 7fbf9dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 0x7fbf98155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.782+0000 7fbf9dc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf981514e0 0x7fbf98155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.782+0000 7fbf9dc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbf980c8140 con 0x7fbf980cb440 2024-04-24T22:40:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.782+0000 7fbf9dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbf980c7e60 con 0x7fbf980c76c0 2024-04-24T22:40:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.782+0000 7fbf9dc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbf9815ab70 con 0x7fbf981514e0 2024-04-24T22:40:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 0x7fbf98155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf980c76c0 0x7fbf981f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 0x7fbf98155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43950/0 (socket says 172.21.15.47:43950) 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97400640 1 -- 172.21.15.47:0/1879400269 learned_addr learned my addr 172.21.15.47:0/1879400269 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf9d200640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf981514e0 0x7fbf98155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97400640 1 -- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf981514e0 msgr2=0x7fbf98155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97400640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf981514e0 0x7fbf98155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97400640 1 -- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf980c76c0 msgr2=0x7fbf981f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97400640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf980c76c0 0x7fbf981f3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97400640 1 -- 172.21.15.47:0/1879400269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf980c8c80 con 0x7fbf980cb440 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97e00640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf980c76c0 0x7fbf981f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf97400640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 0x7fbf98155a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400e870 tx=0x7fbf8400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.783+0000 7fbf8fe00640 1 -- 172.21.15.47:0/1879400269 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84017070 con 0x7fbf980cb440 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.784+0000 7fbf8fe00640 1 -- 172.21.15.47:0/1879400269 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8400c940 con 0x7fbf980cb440 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.784+0000 7fbf9dc00640 1 -- 172.21.15.47:0/1879400269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf981f45e0 con 0x7fbf980cb440 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.784+0000 7fbf8fe00640 1 -- 172.21.15.47:0/1879400269 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8400cd30 con 0x7fbf980cb440 2024-04-24T22:40:04.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.784+0000 7fbf9dc00640 1 -- 172.21.15.47:0/1879400269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf98154730 con 0x7fbf980cb440 2024-04-24T22:40:04.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.785+0000 7fbf9dc00640 1 -- 172.21.15.47:0/1879400269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf980c73c0 con 0x7fbf980cb440 2024-04-24T22:40:04.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.786+0000 7fbf8fe00640 1 -- 172.21.15.47:0/1879400269 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf8401a070 con 0x7fbf980cb440 2024-04-24T22:40:04.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.786+0000 7fbf8fe00640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbf7806be30 0x7fbf7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:04.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.787+0000 7fbf8fe00640 1 -- 172.21.15.47:0/1879400269 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbf8408ad00 con 0x7fbf980cb440 2024-04-24T22:40:04.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.787+0000 7fbf97e00640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbf7806be30 0x7fbf7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:04.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.787+0000 7fbf97e00640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbf7806be30 0x7fbf7806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009560 tx=0x7fbf88009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:04.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:04.791+0000 7fbf8fe00640 1 -- 172.21.15.47:0/1879400269 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf84058fc0 con 0x7fbf980cb440 2024-04-24T22:40:05.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.030+0000 7fbf9dc00640 1 -- 172.21.15.47:0/1879400269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf980c3530 con 0x7fbf980cb440 2024-04-24T22:40:05.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.031+0000 7fbf8fe00640 1 -- 172.21.15.47:0/1879400269 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6912 (secure 0 0 0) 0x7fbf8405d2c0 con 0x7fbf980cb440 2024-04-24T22:40:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:05.034 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1712577517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1712577517},{"type":"v1","addr":"172.21.15.113:6839","nonce":1712577517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:39:37.178621+0000","modified":"2024-04-24T22:40:00.826874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4282,"mds_2":4279,"mds_3":4241,"mds_4":4238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4238":{"gid":4238,"name":"a","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.47:6833/2908847255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2908847255},{"type":"v1","addr":"172.21.15.47:6833","nonce":2908847255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.47:6837/2173854280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2173854280},{"type":"v1","addr":"172.21.15.47:6837","nonce":2173854280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/3103107210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3103107210},{"type":"v1","addr":"172.21.15.113:6837","nonce":3103107210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4282":{"gid":4282,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6838/3621458364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3621458364},{"type":"v1","addr":"172.21.15.113:6838","nonce":3621458364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.47:6835/1117134154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1117134154},{"type":"v1","addr":"172.21.15.47:6835","nonce":1117134154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4282,4279,4306,4241]},"id":1}]} 2024-04-24T22:40:05.034 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 16 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 -- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbf7806be30 msgr2=0x7fbf7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbf7806be30 0x7fbf7806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009560 tx=0x7fbf88009190 comp rx=0 tx=0).stop 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 -- 172.21.15.47:0/1879400269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 msgr2=0x7fbf98155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 0x7fbf98155a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400e870 tx=0x7fbf8400ed30 comp rx=0 tx=0).stop 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 -- 172.21.15.47:0/1879400269 shutdown_connections 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbf7806be30 0x7fbf7806e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf981514e0 0x7fbf98155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf980cb440 0x7fbf98155a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 --2- 172.21.15.47:0/1879400269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf980c76c0 0x7fbf981f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 -- 172.21.15.47:0/1879400269 >> 172.21.15.47:0/1879400269 conn(0x7fbf980c9410 msgr2=0x7fbf98150690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 -- 172.21.15.47:0/1879400269 shutdown_connections 2024-04-24T22:40:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.034+0000 7fbf9dc00640 1 -- 172.21.15.47:0/1879400269 wait complete. 2024-04-24T22:40:05.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:40:05.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.241+0000 7fe2bc600640 1 Processor -- start 2024-04-24T22:40:05.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.241+0000 7fe2bc600640 1 -- start start 2024-04-24T22:40:05.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.243+0000 7fe2bc600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe2b40cbde0 0x7fe2b40ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.243+0000 7fe2bc600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe2b4073380 0x7fe2b40ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.243+0000 7fe2bc600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b415b240 con 0x7fe2b40cc840 2024-04-24T22:40:05.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.243+0000 7fe2bc600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe2b415daa0 con 0x7fe2b40cbde0 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.243+0000 7fe2bc600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe2b415dcf0 con 0x7fe2b4073380 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.243+0000 7fe2b9c00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe2b40cc840 0x7fe2b40cb6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44188/0 (socket says 172.21.15.47:44188) 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.243+0000 7fe2b9c00640 1 -- 172.21.15.47:0/2390444037 learned_addr learned my addr 172.21.15.47:0/2390444037 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.243+0000 7fe2b9200640 1 --2- 172.21.15.47:0/2390444037 >> v2:172.21.15.47:3300/0 conn(0x7fe2b40cbde0 0x7fe2b40ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.244+0000 7fe2ba600640 1 --2- 172.21.15.47:0/2390444037 >> v2:172.21.15.113:3300/0 conn(0x7fe2b4073380 0x7fe2b40ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.244+0000 7fe2b3e00640 1 -- 172.21.15.47:0/2390444037 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4028515755 0 0) 0x7fe2b415b240 con 0x7fe2b40cc840 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.244+0000 7fe2b3e00640 1 -- 172.21.15.47:0/2390444037 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe29c003520 con 0x7fe2b40cc840 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.244+0000 7fe2b9200640 1 -- 172.21.15.47:0/2390444037 >> v2:172.21.15.113:3300/0 conn(0x7fe2b4073380 msgr2=0x7fe2b40ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.244+0000 7fe2b9200640 1 --2- 172.21.15.47:0/2390444037 >> v2:172.21.15.113:3300/0 conn(0x7fe2b4073380 0x7fe2b40ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.244+0000 7fe2b9200640 1 -- 172.21.15.47:0/2390444037 >> v1:172.21.15.47:6789/0 conn(0x7fe2b40cc840 legacy=0x7fe2b40cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.244+0000 7fe2b9200640 1 -- 172.21.15.47:0/2390444037 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b415dfd0 con 0x7fe2b40cbde0 2024-04-24T22:40:05.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.245+0000 7fe2b9200640 1 --2- 172.21.15.47:0/2390444037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cbde0 0x7fe2b40ca7d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe2b408d640 tx=0x7fe2a002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787f8f4d1b35971a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:05.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.245+0000 7fe2b3e00640 1 -- 172.21.15.47:0/2390444037 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0037070 con 0x7fe2b40cbde0 2024-04-24T22:40:05.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.245+0000 7fe2b3e00640 1 -- 172.21.15.47:0/2390444037 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a0032440 con 0x7fe2b40cbde0 2024-04-24T22:40:05.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.245+0000 7fe2b3e00640 1 -- 172.21.15.47:0/2390444037 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a002ae60 con 0x7fe2b40cbde0 2024-04-24T22:40:05.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.246+0000 7fe2bc600640 1 -- 172.21.15.47:0/2390444037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cbde0 msgr2=0x7fe2b40ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.246+0000 7fe2bc600640 1 --2- 172.21.15.47:0/2390444037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cbde0 0x7fe2b40ca7d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe2b408d640 tx=0x7fe2a002a110 comp rx=0 tx=0).stop 2024-04-24T22:40:05.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.246+0000 7fe2bc600640 1 -- 172.21.15.47:0/2390444037 shutdown_connections 2024-04-24T22:40:05.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.246+0000 7fe2bc600640 1 --2- 172.21.15.47:0/2390444037 >> v2:172.21.15.113:3300/0 conn(0x7fe2b4073380 0x7fe2b40ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.246+0000 7fe2bc600640 1 --2- 172.21.15.47:0/2390444037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cbde0 0x7fe2b40ca7d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.246+0000 7fe2bc600640 1 -- 172.21.15.47:0/2390444037 >> 172.21.15.47:0/2390444037 conn(0x7fe2b406c940 msgr2=0x7fe2b4072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:05.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.246+0000 7fe2bc600640 1 -- 172.21.15.47:0/2390444037 shutdown_connections 2024-04-24T22:40:05.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.246+0000 7fe2bc600640 1 -- 172.21.15.47:0/2390444037 wait complete. 2024-04-24T22:40:05.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.248+0000 7fe2bc600640 1 Processor -- start 2024-04-24T22:40:05.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.248+0000 7fe2bc600640 1 -- start start 2024-04-24T22:40:05.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.249+0000 7fe2bc600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b4073380 0x7fe2b40c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.249+0000 7fe2bc600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b40cbde0 0x7fe2b40c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.249+0000 7fe2bc600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cc840 0x7fe2b40cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.249+0000 7fe2bc600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b415e870 con 0x7fe2b40cc840 2024-04-24T22:40:05.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.249+0000 7fe2bc600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b415e590 con 0x7fe2b40cbde0 2024-04-24T22:40:05.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.249+0000 7fe2bc600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b415af30 con 0x7fe2b4073380 2024-04-24T22:40:05.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2ba600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cc840 0x7fe2b40cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2ba600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cc840 0x7fe2b40cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43972/0 (socket says 172.21.15.47:43972) 2024-04-24T22:40:05.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2ba600640 1 -- 172.21.15.47:0/3124422044 learned_addr learned my addr 172.21.15.47:0/3124422044 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:05.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2b9200640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b40cbde0 0x7fe2b40c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2b9c00640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b4073380 0x7fe2b40c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2ba600640 1 -- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b4073380 msgr2=0x7fe2b40c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2ba600640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b4073380 0x7fe2b40c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2ba600640 1 -- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b40cbde0 msgr2=0x7fe2b40c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2ba600640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b40cbde0 0x7fe2b40c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2ba600640 1 -- 172.21.15.47:0/3124422044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b4153820 con 0x7fe2b40cc840 2024-04-24T22:40:05.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.250+0000 7fe2ba600640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cc840 0x7fe2b40cdf80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00e820 tx=0x7fe2ac00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.251+0000 7fe2b1600640 1 -- 172.21.15.47:0/3124422044 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac017070 con 0x7fe2b40cc840 2024-04-24T22:40:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.251+0000 7fe2b1600640 1 -- 172.21.15.47:0/3124422044 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2ac00ee70 con 0x7fe2b40cc840 2024-04-24T22:40:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.251+0000 7fe2b1600640 1 -- 172.21.15.47:0/3124422044 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac00cb60 con 0x7fe2b40cc840 2024-04-24T22:40:05.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.251+0000 7fe2bc600640 1 -- 172.21.15.47:0/3124422044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b40cea80 con 0x7fe2b40cc840 2024-04-24T22:40:05.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.251+0000 7fe2bc600640 1 -- 172.21.15.47:0/3124422044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b40c8e40 con 0x7fe2b40cc840 2024-04-24T22:40:05.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.252+0000 7fe2bc600640 1 -- 172.21.15.47:0/3124422044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b4073b90 con 0x7fe2b40cc840 2024-04-24T22:40:05.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.253+0000 7fe2b1600640 1 -- 172.21.15.47:0/3124422044 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2ac01a070 con 0x7fe2b40cc840 2024-04-24T22:40:05.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.255+0000 7fe2b1600640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe29406be30 0x7fe29406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.255+0000 7fe2b1600640 1 -- 172.21.15.47:0/3124422044 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe2ac08b010 con 0x7fe2b40cc840 2024-04-24T22:40:05.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.255+0000 7fe2b9c00640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe29406be30 0x7fe29406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.256+0000 7fe2b9c00640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe29406be30 0x7fe29406e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8005c00 tx=0x7fe2a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:05.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.259+0000 7fe2b1600640 1 -- 172.21.15.47:0/3124422044 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe2ac059370 con 0x7fe2b40cc840 2024-04-24T22:40:05.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.456+0000 7fe2bc600640 1 -- 172.21.15.47:0/3124422044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2b40cadb0 con 0x7fe2b40cc840 2024-04-24T22:40:05.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.458+0000 7fe2b1600640 1 -- 172.21.15.47:0/3124422044 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16593 (secure 0 0 0) 0x7fe2ac05d670 con 0x7fe2b40cc840 2024-04-24T22:40:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:05.461 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:38.692613+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:40:05.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.460+0000 7fe2bc600640 1 -- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe29406be30 msgr2=0x7fe29406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.460+0000 7fe2bc600640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe29406be30 0x7fe29406e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8005c00 tx=0x7fe2a8009040 comp rx=0 tx=0).stop 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.460+0000 7fe2bc600640 1 -- 172.21.15.47:0/3124422044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cc840 msgr2=0x7fe2b40cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.460+0000 7fe2bc600640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cc840 0x7fe2b40cdf80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00e820 tx=0x7fe2ac00ec20 comp rx=0 tx=0).stop 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.460+0000 7fe2bc600640 1 -- 172.21.15.47:0/3124422044 shutdown_connections 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.460+0000 7fe2bc600640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe29406be30 0x7fe29406e2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.460+0000 7fe2bc600640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2b40cc840 0x7fe2b40cdf80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.460+0000 7fe2bc600640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b40cbde0 0x7fe2b40c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.460+0000 7fe2bc600640 1 --2- 172.21.15.47:0/3124422044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b4073380 0x7fe2b40c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.460+0000 7fe2bc600640 1 -- 172.21.15.47:0/3124422044 >> 172.21.15.47:0/3124422044 conn(0x7fe2b406c940 msgr2=0x7fe2b4072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.461+0000 7fe2bc600640 1 -- 172.21.15.47:0/3124422044 shutdown_connections 2024-04-24T22:40:05.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.461+0000 7fe2bc600640 1 -- 172.21.15.47:0/3124422044 wait complete. 2024-04-24T22:40:05.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:30.711538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:34.803185+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:36.184123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:40:05.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:40:05.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.676+0000 7f7862600640 1 Processor -- start 2024-04-24T22:40:05.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.677+0000 7f7862600640 1 -- start start 2024-04-24T22:40:05.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.678+0000 7f7862600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c154560 0x7f785c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.678+0000 7f7862600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f785c0cb970 0x7f785c0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.678+0000 7f7862600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f785c0cbe00 con 0x7f785c073280 2024-04-24T22:40:05.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.678+0000 7f7862600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f785c15b2c0 con 0x7f785c0cb970 2024-04-24T22:40:05.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.678+0000 7f7862600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f785c15da90 con 0x7f785c154560 2024-04-24T22:40:05.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.679+0000 7f7861200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c154560 0x7f785c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.679+0000 7f785be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f785c073280 0x7f785c0cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44196/0 (socket says 172.21.15.47:44196) 2024-04-24T22:40:05.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.679+0000 7f785be00640 1 -- 172.21.15.47:0/3223626859 learned_addr learned my addr 172.21.15.47:0/3223626859 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.679+0000 7f7861c00640 1 --2- 172.21.15.47:0/3223626859 >> v2:172.21.15.47:3300/0 conn(0x7f785c0cb970 0x7f785c0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.679+0000 7f7861200640 1 -- 172.21.15.47:0/3223626859 >> v2:172.21.15.47:3300/0 conn(0x7f785c0cb970 msgr2=0x7f785c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.679+0000 7f7861200640 1 --2- 172.21.15.47:0/3223626859 >> v2:172.21.15.47:3300/0 conn(0x7f785c0cb970 0x7f785c0ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.679+0000 7f7861200640 1 -- 172.21.15.47:0/3223626859 >> v1:172.21.15.47:6789/0 conn(0x7f785c073280 legacy=0x7f785c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.679+0000 7f7861200640 1 -- 172.21.15.47:0/3223626859 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c15dd70 con 0x7f785c154560 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.680+0000 7f7861200640 1 --2- 172.21.15.47:0/3223626859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c154560 0x7f785c0cb430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f78500097a0 tx=0x7f785002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf8aa8465233898b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f785b400640 1 -- 172.21.15.47:0/3223626859 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850037070 con 0x7f785c154560 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f785b400640 1 -- 172.21.15.47:0/3223626859 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7850032440 con 0x7f785c154560 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f785b400640 1 -- 172.21.15.47:0/3223626859 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850032810 con 0x7f785c154560 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f7862600640 1 -- 172.21.15.47:0/3223626859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c154560 msgr2=0x7f785c0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f7862600640 1 --2- 172.21.15.47:0/3223626859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c154560 0x7f785c0cb430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f78500097a0 tx=0x7f785002a180 comp rx=0 tx=0).stop 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f7862600640 1 -- 172.21.15.47:0/3223626859 shutdown_connections 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f7862600640 1 --2- 172.21.15.47:0/3223626859 >> v2:172.21.15.47:3300/0 conn(0x7f785c0cb970 0x7f785c0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f7862600640 1 --2- 172.21.15.47:0/3223626859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c154560 0x7f785c0cb430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f7862600640 1 -- 172.21.15.47:0/3223626859 >> 172.21.15.47:0/3223626859 conn(0x7f785c0672c0 msgr2=0x7f785c072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f7862600640 1 -- 172.21.15.47:0/3223626859 shutdown_connections 2024-04-24T22:40:05.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.681+0000 7f7862600640 1 -- 172.21.15.47:0/3223626859 wait complete. 2024-04-24T22:40:05.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.683+0000 7f7862600640 1 Processor -- start 2024-04-24T22:40:05.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.683+0000 7f7862600640 1 -- start start 2024-04-24T22:40:05.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f7862600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f785c073280 0x7f785c156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f7862600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c0cb970 0x7f785c15be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f7862600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f785c154560 0x7f785c15c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f7862600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f785c0caad0 con 0x7f785c073280 2024-04-24T22:40:05.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f7862600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f785c15e330 con 0x7f785c154560 2024-04-24T22:40:05.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f7862600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f785c0cd2b0 con 0x7f785c0cb970 2024-04-24T22:40:05.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f7861200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f785c073280 0x7f785c156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f785be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c0cb970 0x7f785c15be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f7861200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f785c073280 0x7f785c156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43996/0 (socket says 172.21.15.47:43996) 2024-04-24T22:40:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f7861200640 1 -- 172.21.15.47:0/2929154992 learned_addr learned my addr 172.21.15.47:0/2929154992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.684+0000 7f7861c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f785c154560 0x7f785c15c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.685+0000 7f7861200640 1 -- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c0cb970 msgr2=0x7f785c15be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.685+0000 7f7861200640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c0cb970 0x7f785c15be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.685+0000 7f7861200640 1 -- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f785c154560 msgr2=0x7f785c15c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.685+0000 7f7861200640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f785c154560 0x7f785c15c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.685+0000 7f7861200640 1 -- 172.21.15.47:0/2929154992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c154e90 con 0x7f785c073280 2024-04-24T22:40:05.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.685+0000 7f7861200640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f785c073280 0x7f785c156d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f78500097a0 tx=0x7f78500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:05.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.685+0000 7f7858c00640 1 -- 172.21.15.47:0/2929154992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850041070 con 0x7f785c073280 2024-04-24T22:40:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.685+0000 7f7858c00640 1 -- 172.21.15.47:0/2929154992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785003c870 con 0x7f785c073280 2024-04-24T22:40:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.685+0000 7f7858c00640 1 -- 172.21.15.47:0/2929154992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850037040 con 0x7f785c073280 2024-04-24T22:40:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.685+0000 7f7862600640 1 -- 172.21.15.47:0/2929154992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785c155130 con 0x7f785c073280 2024-04-24T22:40:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.686+0000 7f7862600640 1 -- 172.21.15.47:0/2929154992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785c15b660 con 0x7f785c073280 2024-04-24T22:40:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.687+0000 7f7862600640 1 -- 172.21.15.47:0/2929154992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f785c14e5e0 con 0x7f785c073280 2024-04-24T22:40:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.688+0000 7f7858c00640 1 -- 172.21.15.47:0/2929154992 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7850044070 con 0x7f785c073280 2024-04-24T22:40:05.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.688+0000 7f7858c00640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f783c06be60 0x7f783c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:05.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.689+0000 7f7858c00640 1 -- 172.21.15.47:0/2929154992 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f78500aba00 con 0x7f785c073280 2024-04-24T22:40:05.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.689+0000 7f785be00640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f783c06be60 0x7f783c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:05.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.690+0000 7f785be00640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f783c06be60 0x7f783c06e320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f785c0c4e10 tx=0x7f7848007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.693+0000 7f7858c00640 1 -- 172.21.15.47:0/2929154992 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f785c14e5e0 con 0x7f785c073280 2024-04-24T22:40:05.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.898+0000 7f7862600640 1 -- 172.21.15.47:0/2929154992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f785c157d20 con 0x7f785c073280 2024-04-24T22:40:05.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.899+0000 7f7858c00640 1 -- 172.21.15.47:0/2929154992 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f7850079df0 con 0x7f785c073280 2024-04-24T22:40:05.901 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-24T22:40:05.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 -- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f783c06be60 msgr2=0x7f783c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f783c06be60 0x7f783c06e320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f785c0c4e10 tx=0x7f7848007e80 comp rx=0 tx=0).stop 2024-04-24T22:40:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 -- 172.21.15.47:0/2929154992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f785c073280 msgr2=0x7f785c156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f785c073280 0x7f785c156d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f78500097a0 tx=0x7f78500046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 -- 172.21.15.47:0/2929154992 shutdown_connections 2024-04-24T22:40:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f783c06be60 0x7f783c06e320 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f785c154560 0x7f785c15c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f785c0cb970 0x7f785c15be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 --2- 172.21.15.47:0/2929154992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f785c073280 0x7f785c156d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 -- 172.21.15.47:0/2929154992 >> 172.21.15.47:0/2929154992 conn(0x7f785c0672c0 msgr2=0x7f785c1510c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 -- 172.21.15.47:0/2929154992 shutdown_connections 2024-04-24T22:40:05.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:05.902+0000 7f7862600640 1 -- 172.21.15.47:0/2929154992 wait complete. 2024-04-24T22:40:05.924 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:40:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.123+0000 7f8818e00640 1 Processor -- start 2024-04-24T22:40:06.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.123+0000 7f8818e00640 1 -- start start 2024-04-24T22:40:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.125+0000 7f8818e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88140cbb40 0x7f88140cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.125+0000 7f8818e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8814073280 0x7f88140ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.125+0000 7f8818e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881415b2c0 con 0x7f8814154560 2024-04-24T22:40:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.125+0000 7f8818e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f881415da90 con 0x7f8814073280 2024-04-24T22:40:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.125+0000 7f8818e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f881415dd70 con 0x7f88140cbb40 2024-04-24T22:40:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.126+0000 7f8813e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8814073280 0x7f88140ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.126+0000 7f8813e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8814073280 0x7f88140ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57742/0 (socket says 172.21.15.47:57742) 2024-04-24T22:40:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.126+0000 7f8813e00640 1 -- 172.21.15.47:0/1274708600 learned_addr learned my addr 172.21.15.47:0/1274708600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:06.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.126+0000 7f880aa00640 1 --2- 172.21.15.47:0/1274708600 >> v2:172.21.15.113:3300/0 conn(0x7f88140cbb40 0x7f88140cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.126+0000 7f8813e00640 1 -- 172.21.15.47:0/1274708600 >> v2:172.21.15.113:3300/0 conn(0x7f88140cbb40 msgr2=0x7f88140cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.126+0000 7f8813e00640 1 --2- 172.21.15.47:0/1274708600 >> v2:172.21.15.113:3300/0 conn(0x7f88140cbb40 0x7f88140cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.126+0000 7f8813e00640 1 -- 172.21.15.47:0/1274708600 >> v1:172.21.15.113:6789/0 conn(0x7f8814154560 legacy=0x7f88140cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.126+0000 7f8813e00640 1 -- 172.21.15.47:0/1274708600 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881415e050 con 0x7f8814073280 2024-04-24T22:40:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.127+0000 7f8813e00640 1 --2- 172.21.15.47:0/1274708600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8814073280 0x7f88140ccba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f881406c5d0 tx=0x7f880c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2c8eb7b1eb30ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.128+0000 7f8812a00640 1 -- 172.21.15.47:0/1274708600 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c036070 con 0x7f8814073280 2024-04-24T22:40:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.128+0000 7f8812a00640 1 -- 172.21.15.47:0/1274708600 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c031600 con 0x7f8814073280 2024-04-24T22:40:06.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.128+0000 7f8812a00640 1 -- 172.21.15.47:0/1274708600 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c0319d0 con 0x7f8814073280 2024-04-24T22:40:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.128+0000 7f8818e00640 1 -- 172.21.15.47:0/1274708600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8814073280 msgr2=0x7f88140ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.128+0000 7f8818e00640 1 --2- 172.21.15.47:0/1274708600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8814073280 0x7f88140ccba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f881406c5d0 tx=0x7f880c02a6a0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.128+0000 7f8818e00640 1 -- 172.21.15.47:0/1274708600 shutdown_connections 2024-04-24T22:40:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.128+0000 7f8818e00640 1 --2- 172.21.15.47:0/1274708600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8814073280 0x7f88140ccba0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.128+0000 7f8818e00640 1 --2- 172.21.15.47:0/1274708600 >> v2:172.21.15.113:3300/0 conn(0x7f88140cbb40 0x7f88140cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.128+0000 7f8818e00640 1 -- 172.21.15.47:0/1274708600 >> 172.21.15.47:0/1274708600 conn(0x7f88140672c0 msgr2=0x7f8814072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.129+0000 7f8818e00640 1 -- 172.21.15.47:0/1274708600 shutdown_connections 2024-04-24T22:40:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.129+0000 7f8818e00640 1 -- 172.21.15.47:0/1274708600 wait complete. 2024-04-24T22:40:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.130+0000 7f8818e00640 1 Processor -- start 2024-04-24T22:40:06.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.130+0000 7f8818e00640 1 -- start start 2024-04-24T22:40:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.131+0000 7f8818e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8814073280 0x7f88141f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.131+0000 7f8818e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88140cbb40 0x7f8814152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.131+0000 7f8818e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8814154560 0x7f88141e53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.131+0000 7f8818e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f881415ae80 con 0x7f88140cbb40 2024-04-24T22:40:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.131+0000 7f8818e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f88140caad0 con 0x7f8814073280 2024-04-24T22:40:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.131+0000 7f8818e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f881415b000 con 0x7f8814154560 2024-04-24T22:40:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.131+0000 7f8813400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8814073280 0x7f88141f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.131+0000 7f880aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88140cbb40 0x7f8814152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.131+0000 7f8813400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8814073280 0x7f88141f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:47528/0 (socket says 172.21.15.47:47528) 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.131+0000 7f8813400640 1 -- 172.21.15.47:0/1925367460 learned_addr learned my addr 172.21.15.47:0/1925367460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f8813e00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8814154560 0x7f88141e53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f8813400640 1 -- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8814154560 msgr2=0x7f88141e53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f8813400640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8814154560 0x7f88141e53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f8813400640 1 -- 172.21.15.47:0/1925367460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88140cbb40 msgr2=0x7f8814152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f8813400640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88140cbb40 0x7f8814152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f8813400640 1 -- 172.21.15.47:0/1925367460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881415d060 con 0x7f8814073280 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f880aa00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88140cbb40 0x7f8814152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f8813400640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8814073280 0x7f88141f4540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f880400c3c0 tx=0x7f880400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f880be00640 1 -- 172.21.15.47:0/1925367460 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88040052d0 con 0x7f8814073280 2024-04-24T22:40:06.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f880be00640 1 -- 172.21.15.47:0/1925367460 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8804005490 con 0x7f8814073280 2024-04-24T22:40:06.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f880be00640 1 -- 172.21.15.47:0/1925367460 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804007260 con 0x7f8814073280 2024-04-24T22:40:06.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.132+0000 7f8818e00640 1 -- 172.21.15.47:0/1925367460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881415d340 con 0x7f8814073280 2024-04-24T22:40:06.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.133+0000 7f8818e00640 1 -- 172.21.15.47:0/1925367460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88141e48e0 con 0x7f8814073280 2024-04-24T22:40:06.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.134+0000 7f8818e00640 1 -- 172.21.15.47:0/1925367460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88141540c0 con 0x7f8814073280 2024-04-24T22:40:06.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.135+0000 7f880be00640 1 -- 172.21.15.47:0/1925367460 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8804007400 con 0x7f8814073280 2024-04-24T22:40:06.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.136+0000 7f880be00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f87f006be30 0x7f87f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.136+0000 7f880be00640 1 -- 172.21.15.47:0/1925367460 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f880408a3c0 con 0x7f8814073280 2024-04-24T22:40:06.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.136+0000 7f880aa00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f87f006be30 0x7f87f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.137+0000 7f880aa00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f87f006be30 0x7f87f006e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f881415c360 tx=0x7f87fc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:06.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.141+0000 7f880be00640 1 -- 172.21.15.47:0/1925367460 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f88141540c0 con 0x7f8814073280 2024-04-24T22:40:06.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.338+0000 7f8818e00640 1 -- 172.21.15.47:0/1925367460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f88140c6080 con 0x7f8814073280 2024-04-24T22:40:06.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.339+0000 7f880be00640 1 -- 172.21.15.47:0/1925367460 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f88040585f0 con 0x7f8814073280 2024-04-24T22:40:06.341 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-24T22:40:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.341+0000 7f8818e00640 1 -- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f87f006be30 msgr2=0x7f87f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.341+0000 7f8818e00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f87f006be30 0x7f87f006e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f881415c360 tx=0x7f87fc009190 comp rx=0 tx=0).stop 2024-04-24T22:40:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.341+0000 7f8818e00640 1 -- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8814073280 msgr2=0x7f88141f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.341+0000 7f8818e00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8814073280 0x7f88141f4540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f880400c3c0 tx=0x7f880400c880 comp rx=0 tx=0).stop 2024-04-24T22:40:06.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.341+0000 7f8818e00640 1 -- 172.21.15.47:0/1925367460 shutdown_connections 2024-04-24T22:40:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.341+0000 7f8818e00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f87f006be30 0x7f87f006e2f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.341+0000 7f8818e00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8814154560 0x7f88141e53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.341+0000 7f8818e00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88140cbb40 0x7f8814152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.341+0000 7f8818e00640 1 --2- 172.21.15.47:0/1925367460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8814073280 0x7f88141f4540 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.341+0000 7f8818e00640 1 -- 172.21.15.47:0/1925367460 >> 172.21.15.47:0/1925367460 conn(0x7f88140672c0 msgr2=0x7f88140cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.342+0000 7f8818e00640 1 -- 172.21.15.47:0/1925367460 shutdown_connections 2024-04-24T22:40:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.342+0000 7f8818e00640 1 -- 172.21.15.47:0/1925367460 wait complete. 2024-04-24T22:40:06.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:40:06.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.568+0000 7fb2e7000640 1 Processor -- start 2024-04-24T22:40:06.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.568+0000 7fb2e7000640 1 -- start start 2024-04-24T22:40:06.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.569+0000 7fb2e7000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e006e1b0 0x7fb2e006e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e7000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2e00707e0 0x7fb2e015e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e7000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e00c6f30 con 0x7fb2e0070ea0 2024-04-24T22:40:06.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e7000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e00c7210 con 0x7fb2e00707e0 2024-04-24T22:40:06.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e7000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e015a500 con 0x7fb2e006e1b0 2024-04-24T22:40:06.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e5200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2e00707e0 0x7fb2e015e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e5c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e006e1b0 0x7fb2e006e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e6600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb2e0070ea0 0x7fb2e006b160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:54246/0 (socket says 172.21.15.47:54246) 2024-04-24T22:40:06.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e6600640 1 -- 172.21.15.47:0/265611174 learned_addr learned my addr 172.21.15.47:0/265611174 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:06.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e5200640 1 -- 172.21.15.47:0/265611174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e006e1b0 msgr2=0x7fb2e006e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:06.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e5200640 1 --2- 172.21.15.47:0/265611174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e006e1b0 0x7fb2e006e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.570+0000 7fb2e5200640 1 -- 172.21.15.47:0/265611174 >> v1:172.21.15.113:6789/0 conn(0x7fb2e0070ea0 legacy=0x7fb2e006b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.571+0000 7fb2e5200640 1 -- 172.21.15.47:0/265611174 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e015a7e0 con 0x7fb2e00707e0 2024-04-24T22:40:06.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.571+0000 7fb2dfe00640 1 -- 172.21.15.47:0/265611174 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1928089890 0 0) 0x7fb2e00c6f30 con 0x7fb2e0070ea0 2024-04-24T22:40:06.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.571+0000 7fb2e5200640 1 --2- 172.21.15.47:0/265611174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e00707e0 0x7fb2e015e3a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0097a0 tx=0x7fb2cc02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33cc69c7133820f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:06.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.572+0000 7fb2dfe00640 1 -- 172.21.15.47:0/265611174 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc037070 con 0x7fb2e00707e0 2024-04-24T22:40:06.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.572+0000 7fb2dfe00640 1 -- 172.21.15.47:0/265611174 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2cc032440 con 0x7fb2e00707e0 2024-04-24T22:40:06.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.572+0000 7fb2dfe00640 1 -- 172.21.15.47:0/265611174 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc032810 con 0x7fb2e00707e0 2024-04-24T22:40:06.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.572+0000 7fb2e7000640 1 -- 172.21.15.47:0/265611174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e00707e0 msgr2=0x7fb2e015e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.572+0000 7fb2e7000640 1 --2- 172.21.15.47:0/265611174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e00707e0 0x7fb2e015e3a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0097a0 tx=0x7fb2cc02a210 comp rx=0 tx=0).stop 2024-04-24T22:40:06.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.572+0000 7fb2e7000640 1 -- 172.21.15.47:0/265611174 shutdown_connections 2024-04-24T22:40:06.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.572+0000 7fb2e7000640 1 --2- 172.21.15.47:0/265611174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e00707e0 0x7fb2e015e3a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.572+0000 7fb2e7000640 1 --2- 172.21.15.47:0/265611174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e006e1b0 0x7fb2e006e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.572+0000 7fb2e7000640 1 -- 172.21.15.47:0/265611174 >> 172.21.15.47:0/265611174 conn(0x7fb2e00cb900 msgr2=0x7fb2e00cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:06.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.573+0000 7fb2e7000640 1 -- 172.21.15.47:0/265611174 shutdown_connections 2024-04-24T22:40:06.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.573+0000 7fb2e7000640 1 -- 172.21.15.47:0/265611174 wait complete. 2024-04-24T22:40:06.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.574+0000 7fb2e7000640 1 Processor -- start 2024-04-24T22:40:06.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.574+0000 7fb2e7000640 1 -- start start 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.575+0000 7fb2e7000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e006e1b0 0x7fb2e01589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.575+0000 7fb2e7000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00707e0 0x7fb2e0158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.575+0000 7fb2e7000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e0070ea0 0x7fb2e01e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.575+0000 7fb2e7000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e015b2c0 con 0x7fb2e00707e0 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.575+0000 7fb2e7000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e015e8e0 con 0x7fb2e0070ea0 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.575+0000 7fb2e7000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e0154950 con 0x7fb2e006e1b0 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.575+0000 7fb2e5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00707e0 0x7fb2e0158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.576+0000 7fb2e5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00707e0 0x7fb2e0158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57770/0 (socket says 172.21.15.47:57770) 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.576+0000 7fb2e5200640 1 -- 172.21.15.47:0/1950823824 learned_addr learned my addr 172.21.15.47:0/1950823824 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.576+0000 7fb2e6600640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e0070ea0 0x7fb2e01e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.576+0000 7fb2e5c00640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e006e1b0 0x7fb2e01589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.576+0000 7fb2e5200640 1 -- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e006e1b0 msgr2=0x7fb2e01589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.576+0000 7fb2e5200640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e006e1b0 0x7fb2e01589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.576+0000 7fb2e5200640 1 -- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e0070ea0 msgr2=0x7fb2e01e52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.576+0000 7fb2e5200640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e0070ea0 0x7fb2e01e52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.576+0000 7fb2e5200640 1 -- 172.21.15.47:0/1950823824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e00c3f60 con 0x7fb2e00707e0 2024-04-24T22:40:06.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.576+0000 7fb2e5200640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00707e0 0x7fb2e0158f30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc035040 tx=0x7fb2cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:06.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.577+0000 7fb2dd600640 1 -- 172.21.15.47:0/1950823824 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc037040 con 0x7fb2e00707e0 2024-04-24T22:40:06.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.577+0000 7fb2dd600640 1 -- 172.21.15.47:0/1950823824 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2cc030500 con 0x7fb2e00707e0 2024-04-24T22:40:06.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.577+0000 7fb2dd600640 1 -- 172.21.15.47:0/1950823824 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0308d0 con 0x7fb2e00707e0 2024-04-24T22:40:06.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.577+0000 7fb2e7000640 1 -- 172.21.15.47:0/1950823824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e014c510 con 0x7fb2e00707e0 2024-04-24T22:40:06.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.577+0000 7fb2e7000640 1 -- 172.21.15.47:0/1950823824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e00c39a0 con 0x7fb2e00707e0 2024-04-24T22:40:06.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.578+0000 7fb2e7000640 1 -- 172.21.15.47:0/1950823824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e0068d50 con 0x7fb2e00707e0 2024-04-24T22:40:06.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.579+0000 7fb2dd600640 1 -- 172.21.15.47:0/1950823824 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2cc044020 con 0x7fb2e00707e0 2024-04-24T22:40:06.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.580+0000 7fb2dd600640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 0x7fb2c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.580+0000 7fb2dd600640 1 -- 172.21.15.47:0/1950823824 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb2cc0aafe0 con 0x7fb2e00707e0 2024-04-24T22:40:06.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.580+0000 7fb2e5c00640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 0x7fb2c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.581+0000 7fb2e5c00640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 0x7fb2c006e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009b40 tx=0x7fb2d4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:06.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.584+0000 7fb2dd600640 1 -- 172.21.15.47:0/1950823824 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb2cc0792e0 con 0x7fb2e00707e0 2024-04-24T22:40:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.776+0000 7fb2e7000640 1 -- 172.21.15.47:0/1950823824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb2e00c7450 con 0x7fb2e00707e0 2024-04-24T22:40:06.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.777+0000 7fb2dd600640 1 -- 172.21.15.47:0/1950823824 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fb2cc07d5e0 con 0x7fb2e00707e0 2024-04-24T22:40:06.780 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 -- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 msgr2=0x7fb2c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 0x7fb2c006e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009b40 tx=0x7fb2d4009190 comp rx=0 tx=0).stop 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 -- 172.21.15.47:0/1950823824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00707e0 msgr2=0x7fb2e0158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00707e0 0x7fb2e0158f30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc035040 tx=0x7fb2cc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 -- 172.21.15.47:0/1950823824 shutdown_connections 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 0x7fb2c006e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e0070ea0 0x7fb2e01e52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00707e0 0x7fb2e0158f30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 --2- 172.21.15.47:0/1950823824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e006e1b0 0x7fb2e01589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 -- 172.21.15.47:0/1950823824 >> 172.21.15.47:0/1950823824 conn(0x7fb2e00cb900 msgr2=0x7fb2e00cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 -- 172.21.15.47:0/1950823824 shutdown_connections 2024-04-24T22:40:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.780+0000 7fb2e7000640 1 -- 172.21.15.47:0/1950823824 wait complete. 2024-04-24T22:40:06.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:40:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.990+0000 7fdde2600640 1 Processor -- start 2024-04-24T22:40:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.990+0000 7fdde2600640 1 -- start start 2024-04-24T22:40:06.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.992+0000 7fdde2600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdddc06f270 0x7fdddc0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.992+0000 7fdde2600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc0c8eb0 0x7fdddc158610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.992+0000 7fdde2600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdddc161e40 con 0x7fdddc06ee90 2024-04-24T22:40:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.992+0000 7fdde2600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdddc162140 con 0x7fdddc06f270 2024-04-24T22:40:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.992+0000 7fdde2600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc15f7c0 con 0x7fdddc0c8eb0 2024-04-24T22:40:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.992+0000 7fdde1200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdddc06ee90 0x7fdddc1625d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51942/0 (socket says 172.21.15.47:51942) 2024-04-24T22:40:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.992+0000 7fdde1200640 1 -- 172.21.15.47:0/4190978840 learned_addr learned my addr 172.21.15.47:0/4190978840 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.993+0000 7fdddbe00640 1 --2- 172.21.15.47:0/4190978840 >> v2:172.21.15.113:3300/0 conn(0x7fdddc06f270 0x7fdddc0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.993+0000 7fdde1c00640 1 --2- 172.21.15.47:0/4190978840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc0c8eb0 0x7fdddc158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.993+0000 7fdddb400640 1 -- 172.21.15.47:0/4190978840 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1039199770 0 0) 0x7fdddc161e40 con 0x7fdddc06ee90 2024-04-24T22:40:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.993+0000 7fdddb400640 1 -- 172.21.15.47:0/4190978840 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc4003520 con 0x7fdddc06ee90 2024-04-24T22:40:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.993+0000 7fdde1c00640 1 -- 172.21.15.47:0/4190978840 >> v2:172.21.15.113:3300/0 conn(0x7fdddc06f270 msgr2=0x7fdddc0c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.993+0000 7fdde1c00640 1 --2- 172.21.15.47:0/4190978840 >> v2:172.21.15.113:3300/0 conn(0x7fdddc06f270 0x7fdddc0c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.993+0000 7fdde1c00640 1 -- 172.21.15.47:0/4190978840 >> v1:172.21.15.47:6789/0 conn(0x7fdddc06ee90 legacy=0x7fdddc1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.993+0000 7fdde1c00640 1 -- 172.21.15.47:0/4190978840 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc162ce0 con 0x7fdddc0c8eb0 2024-04-24T22:40:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.993+0000 7fdddbe00640 1 --2- 172.21.15.47:0/4190978840 >> v2:172.21.15.113:3300/0 conn(0x7fdddc06f270 0x7fdddc0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.994+0000 7fdde1c00640 1 --2- 172.21.15.47:0/4190978840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc0c8eb0 0x7fdddc158610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdddc1622c0 tx=0x7fddd402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a0ed770208b4197 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdddb400640 1 -- 172.21.15.47:0/4190978840 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4007740 con 0x7fdddc0c8eb0 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdddb400640 1 -- 172.21.15.47:0/4190978840 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd40078e0 con 0x7fdddc0c8eb0 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdddb400640 1 -- 172.21.15.47:0/4190978840 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4007cb0 con 0x7fdddc0c8eb0 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdde2600640 1 -- 172.21.15.47:0/4190978840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc0c8eb0 msgr2=0x7fdddc158610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdde2600640 1 --2- 172.21.15.47:0/4190978840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc0c8eb0 0x7fdddc158610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdddc1622c0 tx=0x7fddd402a6d0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdde2600640 1 -- 172.21.15.47:0/4190978840 shutdown_connections 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdde2600640 1 --2- 172.21.15.47:0/4190978840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc0c8eb0 0x7fdddc158610 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdde2600640 1 --2- 172.21.15.47:0/4190978840 >> v2:172.21.15.113:3300/0 conn(0x7fdddc06f270 0x7fdddc0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdde2600640 1 -- 172.21.15.47:0/4190978840 >> 172.21.15.47:0/4190978840 conn(0x7fdddc06cf20 msgr2=0x7fdddc06d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdde2600640 1 -- 172.21.15.47:0/4190978840 shutdown_connections 2024-04-24T22:40:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.995+0000 7fdde2600640 1 -- 172.21.15.47:0/4190978840 wait complete. 2024-04-24T22:40:06.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.997+0000 7fdde2600640 1 Processor -- start 2024-04-24T22:40:06.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.997+0000 7fdde2600640 1 -- start start 2024-04-24T22:40:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.998+0000 7fdde2600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc06ee90 0x7fdddc15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.998+0000 7fdde2600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdddc06f270 0x7fdddc158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.998+0000 7fdde2600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc0c8eb0 0x7fdddc142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.998+0000 7fdde2600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc0c7720 con 0x7fdddc06f270 2024-04-24T22:40:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.998+0000 7fdde2600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc0c7440 con 0x7fdddc0c8eb0 2024-04-24T22:40:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.998+0000 7fdde2600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc0c7a00 con 0x7fdddc06ee90 2024-04-24T22:40:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.998+0000 7fdddbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdddc06f270 0x7fdddc158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.998+0000 7fdddbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdddc06f270 0x7fdddc158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57776/0 (socket says 172.21.15.47:57776) 2024-04-24T22:40:07.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.998+0000 7fdddbe00640 1 -- 172.21.15.47:0/3577544615 learned_addr learned my addr 172.21.15.47:0/3577544615 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.998+0000 7fdde1200640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc06ee90 0x7fdddc15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fdde1c00640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc0c8eb0 0x7fdddc142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fdddbe00640 1 -- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc06ee90 msgr2=0x7fdddc15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fdddbe00640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc06ee90 0x7fdddc15ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fdddbe00640 1 -- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc0c8eb0 msgr2=0x7fdddc142990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fdddbe00640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc0c8eb0 0x7fdddc142990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fdddbe00640 1 -- 172.21.15.47:0/3577544615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc1f42c0 con 0x7fdddc06f270 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fdddbe00640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdddc06f270 0x7fdddc158d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fddc800ebe0 tx=0x7fddc800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fddd8c00640 1 -- 172.21.15.47:0/3577544615 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8017070 con 0x7fdddc06f270 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fddd8c00640 1 -- 172.21.15.47:0/3577544615 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc800cc30 con 0x7fdddc06f270 2024-04-24T22:40:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fddd8c00640 1 -- 172.21.15.47:0/3577544615 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8013470 con 0x7fdddc06f270 2024-04-24T22:40:07.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:06.999+0000 7fdde2600640 1 -- 172.21.15.47:0/3577544615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc0cc9d0 con 0x7fdddc06f270 2024-04-24T22:40:07.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.000+0000 7fdde2600640 1 -- 172.21.15.47:0/3577544615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc1f4560 con 0x7fdddc06f270 2024-04-24T22:40:07.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.001+0000 7fdde2600640 1 -- 172.21.15.47:0/3577544615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdddc1f40b0 con 0x7fdddc06f270 2024-04-24T22:40:07.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.002+0000 7fddd8c00640 1 -- 172.21.15.47:0/3577544615 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddc801a070 con 0x7fdddc06f270 2024-04-24T22:40:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.003+0000 7fddd8c00640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fddbc06be60 0x7fddbc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.003+0000 7fddd8c00640 1 -- 172.21.15.47:0/3577544615 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fddc808b020 con 0x7fdddc06f270 2024-04-24T22:40:07.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.003+0000 7fdde1200640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fddbc06be60 0x7fddbc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.004+0000 7fdde1200640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fddbc06be60 0x7fddbc06e320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fddd0005880 tx=0x7fddd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.007+0000 7fddd8c00640 1 -- 172.21.15.47:0/3577544615 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fddc8059410 con 0x7fdddc06f270 2024-04-24T22:40:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.245+0000 7fdde2600640 1 -- 172.21.15.47:0/3577544615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdddc068520 con 0x7fdddc06f270 2024-04-24T22:40:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.247+0000 7fddd8c00640 1 -- 172.21.15.47:0/3577544615 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6916 (secure 0 0 0) 0x7fddc805d710 con 0x7fdddc06f270 2024-04-24T22:40:07.250 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:07.250 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1712577517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1712577517},{"type":"v1","addr":"172.21.15.113:6839","nonce":1712577517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:39:37.178621+0000","modified":"2024-04-24T22:40:06.837213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4282,"mds_2":4279,"mds_3":4241,"mds_4":4238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4238":{"gid":4238,"name":"a","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6833/2908847255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2908847255},{"type":"v1","addr":"172.21.15.47:6833","nonce":2908847255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.47:6837/2173854280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2173854280},{"type":"v1","addr":"172.21.15.47:6837","nonce":2173854280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/3103107210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3103107210},{"type":"v1","addr":"172.21.15.113:6837","nonce":3103107210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4282":{"gid":4282,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6838/3621458364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3621458364},{"type":"v1","addr":"172.21.15.113:6838","nonce":3621458364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.47:6835/1117134154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1117134154},{"type":"v1","addr":"172.21.15.47:6835","nonce":1117134154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4238,4241,4306,4279,4282]},"id":1}]} 2024-04-24T22:40:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2024-04-24T22:40:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 -- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fddbc06be60 msgr2=0x7fddbc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fddbc06be60 0x7fddbc06e320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fddd0005880 tx=0x7fddd0009040 comp rx=0 tx=0).stop 2024-04-24T22:40:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 -- 172.21.15.47:0/3577544615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdddc06f270 msgr2=0x7fdddc158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdddc06f270 0x7fdddc158d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fddc800ebe0 tx=0x7fddc800c520 comp rx=0 tx=0).stop 2024-04-24T22:40:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 -- 172.21.15.47:0/3577544615 shutdown_connections 2024-04-24T22:40:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fddbc06be60 0x7fddbc06e320 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdddc0c8eb0 0x7fdddc142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdddc06f270 0x7fdddc158d40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 --2- 172.21.15.47:0/3577544615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdddc06ee90 0x7fdddc15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 -- 172.21.15.47:0/3577544615 >> 172.21.15.47:0/3577544615 conn(0x7fdddc06cf20 msgr2=0x7fdddc0715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 -- 172.21.15.47:0/3577544615 shutdown_connections 2024-04-24T22:40:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.249+0000 7fdde2600640 1 -- 172.21.15.47:0/3577544615 wait complete. 2024-04-24T22:40:07.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:40:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.453+0000 7fb391200640 1 Processor -- start 2024-04-24T22:40:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.453+0000 7fb391200640 1 -- start start 2024-04-24T22:40:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb391200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb38c0c76c0 0x7fb38c0caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb391200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 0x7fb38c0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb391200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb38c15aec0 con 0x7fb38c1514e0 2024-04-24T22:40:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb391200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb38c1518c0 con 0x7fb38c0c76c0 2024-04-24T22:40:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb391200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb38c0c7c10 con 0x7fb38c0cb440 2024-04-24T22:40:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb38aa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb38c0c76c0 0x7fb38c0caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb38aa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb38c0c76c0 0x7fb38c0caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57784/0 (socket says 172.21.15.47:57784) 2024-04-24T22:40:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb38aa00640 1 -- 172.21.15.47:0/3132265139 learned_addr learned my addr 172.21.15.47:0/3132265139 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb38be00640 1 --2- 172.21.15.47:0/3132265139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 0x7fb38c0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb38aa00640 1 -- 172.21.15.47:0/3132265139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 msgr2=0x7fb38c0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb38aa00640 1 --2- 172.21.15.47:0/3132265139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 0x7fb38c0c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb38aa00640 1 -- 172.21.15.47:0/3132265139 >> v1:172.21.15.113:6789/0 conn(0x7fb38c1514e0 legacy=0x7fb38c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb38aa00640 1 -- 172.21.15.47:0/3132265139 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38c15b1a0 con 0x7fb38c0c76c0 2024-04-24T22:40:07.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.455+0000 7fb38be00640 1 --2- 172.21.15.47:0/3132265139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 0x7fb38c0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:40:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.456+0000 7fb38aa00640 1 --2- 172.21.15.47:0/3132265139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 0x7fb38c0caf00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb374009870 tx=0x7fb37402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6c489560734396 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.457+0000 7fb38a000640 1 -- 172.21.15.47:0/3132265139 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374037070 con 0x7fb38c0c76c0 2024-04-24T22:40:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.457+0000 7fb38a000640 1 -- 172.21.15.47:0/3132265139 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb37402ae50 con 0x7fb38c0c76c0 2024-04-24T22:40:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.457+0000 7fb38a000640 1 -- 172.21.15.47:0/3132265139 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374032860 con 0x7fb38c0c76c0 2024-04-24T22:40:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.457+0000 7fb391200640 1 -- 172.21.15.47:0/3132265139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 msgr2=0x7fb38c0caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.457+0000 7fb391200640 1 --2- 172.21.15.47:0/3132265139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 0x7fb38c0caf00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb374009870 tx=0x7fb37402a300 comp rx=0 tx=0).stop 2024-04-24T22:40:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.457+0000 7fb391200640 1 -- 172.21.15.47:0/3132265139 shutdown_connections 2024-04-24T22:40:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.457+0000 7fb391200640 1 --2- 172.21.15.47:0/3132265139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 0x7fb38c0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.457+0000 7fb391200640 1 --2- 172.21.15.47:0/3132265139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 0x7fb38c0caf00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.457+0000 7fb391200640 1 -- 172.21.15.47:0/3132265139 >> 172.21.15.47:0/3132265139 conn(0x7fb38c0c9410 msgr2=0x7fb38c0c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.457+0000 7fb391200640 1 -- 172.21.15.47:0/3132265139 shutdown_connections 2024-04-24T22:40:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.458+0000 7fb391200640 1 -- 172.21.15.47:0/3132265139 wait complete. 2024-04-24T22:40:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.459+0000 7fb391200640 1 Processor -- start 2024-04-24T22:40:07.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.459+0000 7fb391200640 1 -- start start 2024-04-24T22:40:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.460+0000 7fb391200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 0x7fb38c1f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.460+0000 7fb391200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 0x7fb38c154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.460+0000 7fb391200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb38c1514e0 0x7fb38c1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.460+0000 7fb391200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c0c8140 con 0x7fb38c0c76c0 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.460+0000 7fb391200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c0c7e60 con 0x7fb38c1514e0 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.460+0000 7fb391200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb38c0c3960 con 0x7fb38c0cb440 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.460+0000 7fb38b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 0x7fb38c1f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.460+0000 7fb38b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 0x7fb38c1f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57786/0 (socket says 172.21.15.47:57786) 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.460+0000 7fb38b400640 1 -- 172.21.15.47:0/2620543601 learned_addr learned my addr 172.21.15.47:0/2620543601 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb38aa00640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 0x7fb38c154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb38be00640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb38c1514e0 0x7fb38c1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb38b400640 1 -- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 msgr2=0x7fb38c154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb38b400640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 0x7fb38c154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb38b400640 1 -- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb38c1514e0 msgr2=0x7fb38c1554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb38b400640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb38c1514e0 0x7fb38c1554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb38b400640 1 -- 172.21.15.47:0/2620543601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38c157090 con 0x7fb38c0c76c0 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb38b400640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 0x7fb38c1f0b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00c400 tx=0x7fb37c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb383400640 1 -- 172.21.15.47:0/2620543601 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0052d0 con 0x7fb38c0c76c0 2024-04-24T22:40:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb383400640 1 -- 172.21.15.47:0/2620543601 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb37c005490 con 0x7fb38c0c76c0 2024-04-24T22:40:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb383400640 1 -- 172.21.15.47:0/2620543601 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c007260 con 0x7fb38c0c76c0 2024-04-24T22:40:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.461+0000 7fb391200640 1 -- 172.21.15.47:0/2620543601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38c157330 con 0x7fb38c0c76c0 2024-04-24T22:40:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.462+0000 7fb391200640 1 -- 172.21.15.47:0/2620543601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38c1578f0 con 0x7fb38c0c76c0 2024-04-24T22:40:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.463+0000 7fb391200640 1 -- 172.21.15.47:0/2620543601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38c0c3530 con 0x7fb38c0c76c0 2024-04-24T22:40:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.464+0000 7fb383400640 1 -- 172.21.15.47:0/2620543601 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb37c007400 con 0x7fb38c0c76c0 2024-04-24T22:40:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.464+0000 7fb383400640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb36806be30 0x7fb36806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.465+0000 7fb383400640 1 -- 172.21.15.47:0/2620543601 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb37c08a2f0 con 0x7fb38c0c76c0 2024-04-24T22:40:07.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.465+0000 7fb38aa00640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb36806be30 0x7fb36806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.465+0000 7fb38aa00640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb36806be30 0x7fb36806e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb374034000 tx=0x7fb37402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.469+0000 7fb383400640 1 -- 172.21.15.47:0/2620543601 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb37c058650 con 0x7fb38c0c76c0 2024-04-24T22:40:07.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.665+0000 7fb391200640 1 -- 172.21.15.47:0/2620543601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb38c04f5e0 con 0x7fb38c0c76c0 2024-04-24T22:40:07.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.667+0000 7fb383400640 1 -- 172.21.15.47:0/2620543601 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16593 (secure 0 0 0) 0x7fb37c05c950 con 0x7fb38c0c76c0 2024-04-24T22:40:07.670 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:07.671 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:38.692613+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:40:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 -- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb36806be30 msgr2=0x7fb36806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb36806be30 0x7fb36806e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb374034000 tx=0x7fb37402e000 comp rx=0 tx=0).stop 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 -- 172.21.15.47:0/2620543601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 msgr2=0x7fb38c1f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 0x7fb38c1f0b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00c400 tx=0x7fb37c00c8c0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 -- 172.21.15.47:0/2620543601 shutdown_connections 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb36806be30 0x7fb36806e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb38c1514e0 0x7fb38c1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb38c0cb440 0x7fb38c154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 --2- 172.21.15.47:0/2620543601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb38c0c76c0 0x7fb38c1f0b00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 -- 172.21.15.47:0/2620543601 >> 172.21.15.47:0/2620543601 conn(0x7fb38c0c9410 msgr2=0x7fb38c0c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 -- 172.21.15.47:0/2620543601 shutdown_connections 2024-04-24T22:40:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.670+0000 7fb391200640 1 -- 172.21.15.47:0/2620543601 wait complete. 2024-04-24T22:40:07.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:40:07.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.873+0000 7f16fbe00640 1 Processor -- start 2024-04-24T22:40:07.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.873+0000 7f16fbe00640 1 -- start start 2024-04-24T22:40:07.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.875+0000 7f16fbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16fc0c67e0 0x7f16fc0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.875+0000 7f16fbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f16fc0c7d20 0x7f16fc066ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.875+0000 7f16fbe00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16fc0cb690 con 0x7f16fc0c6bc0 2024-04-24T22:40:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.875+0000 7f16fbe00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f16fc15aeb0 con 0x7f16fc0c7d20 2024-04-24T22:40:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.875+0000 7f16fbe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f16fc15b190 con 0x7f16fc0c67e0 2024-04-24T22:40:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.875+0000 7f16fa000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f16fc0c6bc0 0x7f16fc15a590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51958/0 (socket says 172.21.15.47:51958) 2024-04-24T22:40:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.875+0000 7f16fa000640 1 -- 172.21.15.47:0/3349101764 learned_addr learned my addr 172.21.15.47:0/3349101764 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.876+0000 7f16faa00640 1 --2- 172.21.15.47:0/3349101764 >> v2:172.21.15.113:3300/0 conn(0x7f16fc0c67e0 0x7f16fc0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.876+0000 7f16fb400640 1 --2- 172.21.15.47:0/3349101764 >> v2:172.21.15.47:3300/0 conn(0x7f16fc0c7d20 0x7f16fc066ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.876+0000 7f16faa00640 1 -- 172.21.15.47:0/3349101764 >> v2:172.21.15.47:3300/0 conn(0x7f16fc0c7d20 msgr2=0x7f16fc066ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.876+0000 7f16faa00640 1 --2- 172.21.15.47:0/3349101764 >> v2:172.21.15.47:3300/0 conn(0x7f16fc0c7d20 0x7f16fc066ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.876+0000 7f16faa00640 1 -- 172.21.15.47:0/3349101764 >> v1:172.21.15.47:6789/0 conn(0x7f16fc0c6bc0 legacy=0x7f16fc15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.876+0000 7f16f9600640 1 -- 172.21.15.47:0/3349101764 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3434369795 0 0) 0x7f16fc0cb690 con 0x7f16fc0c6bc0 2024-04-24T22:40:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.876+0000 7f16faa00640 1 -- 172.21.15.47:0/3349101764 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc0cb930 con 0x7f16fc0c67e0 2024-04-24T22:40:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.877+0000 7f16faa00640 1 --2- 172.21.15.47:0/3349101764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16fc0c67e0 0x7f16fc0cc130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f16fc067c50 tx=0x7f16ec02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d37f244d32579056 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.877+0000 7f16f9600640 1 -- 172.21.15.47:0/3349101764 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec037070 con 0x7f16fc0c67e0 2024-04-24T22:40:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.877+0000 7f16f9600640 1 -- 172.21.15.47:0/3349101764 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16ec032440 con 0x7f16fc0c67e0 2024-04-24T22:40:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.877+0000 7f16f9600640 1 -- 172.21.15.47:0/3349101764 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec032810 con 0x7f16fc0c67e0 2024-04-24T22:40:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.878+0000 7f16fbe00640 1 -- 172.21.15.47:0/3349101764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16fc0c67e0 msgr2=0x7f16fc0cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.878+0000 7f16fbe00640 1 --2- 172.21.15.47:0/3349101764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16fc0c67e0 0x7f16fc0cc130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f16fc067c50 tx=0x7f16ec02a250 comp rx=0 tx=0).stop 2024-04-24T22:40:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.878+0000 7f16fbe00640 1 -- 172.21.15.47:0/3349101764 shutdown_connections 2024-04-24T22:40:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.878+0000 7f16fbe00640 1 --2- 172.21.15.47:0/3349101764 >> v2:172.21.15.47:3300/0 conn(0x7f16fc0c7d20 0x7f16fc066ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.878+0000 7f16fbe00640 1 --2- 172.21.15.47:0/3349101764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16fc0c67e0 0x7f16fc0cc130 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.878+0000 7f16fbe00640 1 -- 172.21.15.47:0/3349101764 >> 172.21.15.47:0/3349101764 conn(0x7f16fc0c9410 msgr2=0x7f16fc0c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.878+0000 7f16fbe00640 1 -- 172.21.15.47:0/3349101764 shutdown_connections 2024-04-24T22:40:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.878+0000 7f16fbe00640 1 -- 172.21.15.47:0/3349101764 wait complete. 2024-04-24T22:40:07.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.880+0000 7f16fbe00640 1 Processor -- start 2024-04-24T22:40:07.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.880+0000 7f16fbe00640 1 -- start start 2024-04-24T22:40:07.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16fbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16fc0c67e0 0x7f16fc157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16fbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16fc0c6bc0 0x7f16fc1e5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16fc0c7d20 0x7f16fc1e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16fbe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc0cc670 con 0x7f16fc0c7d20 2024-04-24T22:40:07.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16fbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f16fc15b470 con 0x7f16fc0c6bc0 2024-04-24T22:40:07.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16fbe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f16fc0cc7f0 con 0x7f16fc0c67e0 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16fc0c7d20 0x7f16fc1e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16fc0c7d20 0x7f16fc1e5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57796/0 (socket says 172.21.15.47:57796) 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16fb400640 1 -- 172.21.15.47:0/2923775324 learned_addr learned my addr 172.21.15.47:0/2923775324 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16fa000640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16fc0c6bc0 0x7f16fc1e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.881+0000 7f16faa00640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16fc0c67e0 0x7f16fc157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16fb400640 1 -- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16fc0c67e0 msgr2=0x7f16fc157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16fb400640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16fc0c67e0 0x7f16fc157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16fb400640 1 -- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16fc0c6bc0 msgr2=0x7f16fc1e5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16fb400640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16fc0c6bc0 0x7f16fc1e5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16fb400640 1 -- 172.21.15.47:0/2923775324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16fc1f1040 con 0x7f16fc0c7d20 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16fb400640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16fc0c7d20 0x7f16fc1e5f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009d30 tx=0x7f16f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16f2a00640 1 -- 172.21.15.47:0/2923775324 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4018020 con 0x7f16fc0c7d20 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16f2a00640 1 -- 172.21.15.47:0/2923775324 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f4014480 con 0x7f16fc0c7d20 2024-04-24T22:40:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16f2a00640 1 -- 172.21.15.47:0/2923775324 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4014870 con 0x7f16fc0c7d20 2024-04-24T22:40:07.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16fbe00640 1 -- 172.21.15.47:0/2923775324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16fc1f01e0 con 0x7f16fc0c7d20 2024-04-24T22:40:07.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.882+0000 7f16fbe00640 1 -- 172.21.15.47:0/2923775324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16fc1f07a0 con 0x7f16fc0c7d20 2024-04-24T22:40:07.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.884+0000 7f16fbe00640 1 -- 172.21.15.47:0/2923775324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16fc1e5cb0 con 0x7f16fc0c7d20 2024-04-24T22:40:07.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.885+0000 7f16f2a00640 1 -- 172.21.15.47:0/2923775324 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f16f4005d10 con 0x7f16fc0c7d20 2024-04-24T22:40:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.886+0000 7f16f2a00640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f16dc06be60 0x7f16dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.886+0000 7f16f2a00640 1 -- 172.21.15.47:0/2923775324 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f16f408bac0 con 0x7f16fc0c7d20 2024-04-24T22:40:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.886+0000 7f16faa00640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f16dc06be60 0x7f16dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.887+0000 7f16faa00640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f16dc06be60 0x7f16dc06e320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f16ec034000 tx=0x7f16ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:07.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:07.890+0000 7f16f2a00640 1 -- 172.21.15.47:0/2923775324 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f16f4022030 con 0x7f16fc0c7d20 2024-04-24T22:40:08.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.129+0000 7f16fbe00640 1 -- 172.21.15.47:0/2923775324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16fc0cb3a0 con 0x7f16fc0c7d20 2024-04-24T22:40:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.130+0000 7f16f2a00640 1 -- 172.21.15.47:0/2923775324 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6916 (secure 0 0 0) 0x7f16f4010040 con 0x7f16fc0c7d20 2024-04-24T22:40:08.133 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:08.133 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1712577517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1712577517},{"type":"v1","addr":"172.21.15.113:6839","nonce":1712577517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:39:37.178621+0000","modified":"2024-04-24T22:40:06.837213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4282,"mds_2":4279,"mds_3":4241,"mds_4":4238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4238":{"gid":4238,"name":"a","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6833/2908847255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2908847255},{"type":"v1","addr":"172.21.15.47:6833","nonce":2908847255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.47:6837/2173854280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2173854280},{"type":"v1","addr":"172.21.15.47:6837","nonce":2173854280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/3103107210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3103107210},{"type":"v1","addr":"172.21.15.113:6837","nonce":3103107210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4282":{"gid":4282,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6838/3621458364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3621458364},{"type":"v1","addr":"172.21.15.113:6838","nonce":3621458364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.47:6835/1117134154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1117134154},{"type":"v1","addr":"172.21.15.47:6835","nonce":1117134154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4238,4241,4306,4279,4282]},"id":1}]} 2024-04-24T22:40:08.133 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2024-04-24T22:40:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 -- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f16dc06be60 msgr2=0x7f16dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f16dc06be60 0x7f16dc06e320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f16ec034000 tx=0x7f16ec02e000 comp rx=0 tx=0).stop 2024-04-24T22:40:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 -- 172.21.15.47:0/2923775324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16fc0c7d20 msgr2=0x7f16fc1e5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16fc0c7d20 0x7f16fc1e5f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009d30 tx=0x7f16f400c490 comp rx=0 tx=0).stop 2024-04-24T22:40:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 -- 172.21.15.47:0/2923775324 shutdown_connections 2024-04-24T22:40:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f16dc06be60 0x7f16dc06e320 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16fc0c7d20 0x7f16fc1e5f50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16fc0c6bc0 0x7f16fc1e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 --2- 172.21.15.47:0/2923775324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16fc0c67e0 0x7f16fc157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 -- 172.21.15.47:0/2923775324 >> 172.21.15.47:0/2923775324 conn(0x7f16fc0c9410 msgr2=0x7f16fc066ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 -- 172.21.15.47:0/2923775324 shutdown_connections 2024-04-24T22:40:08.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.133+0000 7f16fbe00640 1 -- 172.21.15.47:0/2923775324 wait complete. 2024-04-24T22:40:08.758 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T22:39:37.178621+0000', 'modified': '2024-04-24T22:40:06.837213+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4306, 'mds_1': 4282, 'mds_2': 4279, 'mds_3': 4241, 'mds_4': 4238}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4238': {'gid': 4238, 'name': 'a', 'rank': 4, 'incarnation': 16, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.47:6833/2908847255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 2908847255}, {'type': 'v1', 'addr': '172.21.15.47:6833', 'nonce': 2908847255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4241': {'gid': 4241, 'name': 'e', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.47:6837/2173854280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 2173854280}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 2173854280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.113:6837/3103107210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 3103107210}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3103107210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.113:6838/3621458364', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 3621458364}, {'type': 'v1', 'addr': '172.21.15.113:6838', 'nonce': 3621458364}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4306': {'gid': 4306, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.47:6835/1117134154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 1117134154}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1117134154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4306, 'qdb_cluster': [4238, 4241, 4306, 4279, 4282]} 2024-04-24T22:40:08.758 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-24T22:40:08.758 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T22:40:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.952+0000 7f9f1d600640 1 Processor -- start 2024-04-24T22:40:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.953+0000 7f9f1d600640 1 -- start start 2024-04-24T22:40:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.954+0000 7f9f1d600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f18073280 0x7f9f180ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.954+0000 7f9f1d600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f1815b2c0 con 0x7f9f180cbb40 2024-04-24T22:40:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.954+0000 7f9f1d600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f1815da90 con 0x7f9f18154560 2024-04-24T22:40:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.954+0000 7f9f1d600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9f1815dd70 con 0x7f9f18073280 2024-04-24T22:40:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.955+0000 7f9f17e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9f18154560 0x7f9f180cb430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:54262/0 (socket says 172.21.15.47:54262) 2024-04-24T22:40:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.955+0000 7f9f17e00640 1 -- 172.21.15.47:0/332786661 learned_addr learned my addr 172.21.15.47:0/332786661 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.955+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/332786661 >> v2:172.21.15.47:3300/0 conn(0x7f9f18073280 0x7f9f180ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.955+0000 7f9f16a00640 1 -- 172.21.15.47:0/332786661 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1886191637 0 0) 0x7f9f1815da90 con 0x7f9f18154560 2024-04-24T22:40:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.955+0000 7f9f16a00640 1 -- 172.21.15.47:0/332786661 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ef8003520 con 0x7f9f18154560 2024-04-24T22:40:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.955+0000 7f9f16a00640 1 -- 172.21.15.47:0/332786661 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4007479168 0 0) 0x7f9f1815b2c0 con 0x7f9f180cbb40 2024-04-24T22:40:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.955+0000 7f9f16a00640 1 -- 172.21.15.47:0/332786661 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f1815da90 con 0x7f9f180cbb40 2024-04-24T22:40:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.955+0000 7f9f1cc00640 1 -- 172.21.15.47:0/332786661 >> v1:172.21.15.113:6789/0 conn(0x7f9f18154560 legacy=0x7f9f180cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.955+0000 7f9f1cc00640 1 -- 172.21.15.47:0/332786661 >> v1:172.21.15.47:6789/0 conn(0x7f9f180cbb40 legacy=0x7f9f180cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.955+0000 7f9f1cc00640 1 -- 172.21.15.47:0/332786661 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1815e050 con 0x7f9f18073280 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.956+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/332786661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f18073280 0x7f9f180ccd70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097a0 tx=0x7f9f1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cedd6d591affb30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.957+0000 7f9f16a00640 1 -- 172.21.15.47:0/332786661 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10037070 con 0x7f9f18073280 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.957+0000 7f9f16a00640 1 -- 172.21.15.47:0/332786661 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f10032440 con 0x7f9f18073280 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.957+0000 7f9f16a00640 1 -- 172.21.15.47:0/332786661 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10032810 con 0x7f9f18073280 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.957+0000 7f9f1d600640 1 -- 172.21.15.47:0/332786661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f18073280 msgr2=0x7f9f180ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.957+0000 7f9f1d600640 1 --2- 172.21.15.47:0/332786661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f18073280 0x7f9f180ccd70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097a0 tx=0x7f9f1002a1a0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.957+0000 7f9f1d600640 1 -- 172.21.15.47:0/332786661 shutdown_connections 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.957+0000 7f9f1d600640 1 --2- 172.21.15.47:0/332786661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f18073280 0x7f9f180ccd70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.957+0000 7f9f1d600640 1 -- 172.21.15.47:0/332786661 >> 172.21.15.47:0/332786661 conn(0x7f9f180672c0 msgr2=0x7f9f18072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.957+0000 7f9f1d600640 1 -- 172.21.15.47:0/332786661 shutdown_connections 2024-04-24T22:40:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.957+0000 7f9f1d600640 1 -- 172.21.15.47:0/332786661 wait complete. 2024-04-24T22:40:08.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.959+0000 7f9f1d600640 1 Processor -- start 2024-04-24T22:40:08.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.959+0000 7f9f1d600640 1 -- start start 2024-04-24T22:40:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.960+0000 7f9f1d600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f18073280 0x7f9f181f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.960+0000 7f9f1d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f180cbb40 0x7f9f180b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.960+0000 7f9f1d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f18154560 0x7f9f18151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.960+0000 7f9f1d600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9f180cd2b0 con 0x7f9f180cbb40 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.960+0000 7f9f1d600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9f180caad0 con 0x7f9f18154560 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.960+0000 7f9f1d600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1815ae80 con 0x7f9f18073280 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.960+0000 7f9f17400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f180cbb40 0x7f9f180b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.960+0000 7f9f17400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f180cbb40 0x7f9f180b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57816/0 (socket says 172.21.15.47:57816) 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f17400640 1 -- 172.21.15.47:0/4070460248 learned_addr learned my addr 172.21.15.47:0/4070460248 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f18154560 0x7f9f18151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f17e00640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f18073280 0x7f9f181f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f1cc00640 1 -- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f18073280 msgr2=0x7f9f181f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f18073280 0x7f9f181f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f1cc00640 1 -- 172.21.15.47:0/4070460248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f180cbb40 msgr2=0x7f9f180b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f180cbb40 0x7f9f180b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f1cc00640 1 -- 172.21.15.47:0/4070460248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f181f4930 con 0x7f9f18154560 2024-04-24T22:40:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f17400640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f180cbb40 0x7f9f180b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f18154560 0x7f9f18151ce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f10031000 tx=0x7f9f100046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.961+0000 7f9f0fe00640 1 -- 172.21.15.47:0/4070460248 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10041030 con 0x7f9f18154560 2024-04-24T22:40:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.962+0000 7f9f0fe00640 1 -- 172.21.15.47:0/4070460248 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1003d420 con 0x7f9f18154560 2024-04-24T22:40:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.962+0000 7f9f0fe00640 1 -- 172.21.15.47:0/4070460248 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10037040 con 0x7f9f18154560 2024-04-24T22:40:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.962+0000 7f9f1d600640 1 -- 172.21.15.47:0/4070460248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1815ba00 con 0x7f9f18154560 2024-04-24T22:40:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.962+0000 7f9f1d600640 1 -- 172.21.15.47:0/4070460248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f181f0b10 con 0x7f9f18154560 2024-04-24T22:40:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.964+0000 7f9f0fe00640 1 -- 172.21.15.47:0/4070460248 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f10044070 con 0x7f9f18154560 2024-04-24T22:40:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.965+0000 7f9f0fe00640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9ef406be30 0x7f9ef406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.965+0000 7f9f0fe00640 1 -- 172.21.15.47:0/4070460248 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9f100ab420 con 0x7f9f18154560 2024-04-24T22:40:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.966+0000 7f9f17e00640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9ef406be30 0x7f9ef406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.966+0000 7f9f21a26740 1 Processor -- start 2024-04-24T22:40:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.966+0000 7f9f17e00640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9ef406be30 0x7f9ef406e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9f080011e0 tx=0x7f9f08007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.966+0000 7f9f21a26740 1 -- start start 2024-04-24T22:40:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e636270 0x55e26e651b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e608e00 0x55e26e60ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e60b250 0x55e26e636650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f21a26740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e26e50ced0 con 0x55e26e608e00 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f21a26740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e26e56fdf0 con 0x55e26e636270 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f21a26740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e26e5188d0 con 0x55e26e60b250 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e636270 0x55e26e651b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e636270 0x55e26e651b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:47586/0 (socket says 172.21.15.47:47586) 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f17e00640 1 -- 172.21.15.47:0/455327534 learned_addr learned my addr 172.21.15.47:0/455327534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/455327534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e60b250 0x55e26e636650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.967+0000 7f9f17400640 1 --2- 172.21.15.47:0/455327534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e608e00 0x55e26e60ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.968+0000 7f9f17e00640 1 -- 172.21.15.47:0/455327534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e60b250 msgr2=0x55e26e636650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.968+0000 7f9f17e00640 1 --2- 172.21.15.47:0/455327534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e60b250 0x55e26e636650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.968+0000 7f9f17e00640 1 -- 172.21.15.47:0/455327534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e608e00 msgr2=0x55e26e60ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.968+0000 7f9f17e00640 1 --2- 172.21.15.47:0/455327534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e608e00 0x55e26e60ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.968+0000 7f9f17e00640 1 -- 172.21.15.47:0/455327534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26e2842d0 con 0x55e26e636270 2024-04-24T22:40:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.968+0000 7f9f17e00640 1 --2- 172.21.15.47:0/455327534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e636270 0x55e26e651b10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f08001820 tx=0x7f9f08012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e41d2fdee01654c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.968+0000 7f9f07e00640 1 -- 172.21.15.47:0/455327534 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08025070 con 0x55e26e636270 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.968+0000 7f9f07e00640 1 -- 172.21.15.47:0/455327534 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f0801b050 con 0x55e26e636270 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.968+0000 7f9f07e00640 1 -- 172.21.15.47:0/455327534 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08014e00 con 0x55e26e636270 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.969+0000 7f9f21a26740 1 -- 172.21.15.47:0/455327534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e636270 msgr2=0x55e26e651b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.969+0000 7f9f21a26740 1 --2- 172.21.15.47:0/455327534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e636270 0x55e26e651b10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f08001820 tx=0x7f9f08012000 comp rx=0 tx=0).stop 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.969+0000 7f9f21a26740 1 -- 172.21.15.47:0/455327534 shutdown_connections 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.969+0000 7f9f21a26740 1 --2- 172.21.15.47:0/455327534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e60b250 0x55e26e636650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.969+0000 7f9f21a26740 1 --2- 172.21.15.47:0/455327534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e608e00 0x55e26e60ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.969+0000 7f9f21a26740 1 --2- 172.21.15.47:0/455327534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e636270 0x55e26e651b10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.969+0000 7f9f21a26740 1 -- 172.21.15.47:0/455327534 >> 172.21.15.47:0/455327534 conn(0x55e26e609c90 msgr2=0x55e26e651160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.969+0000 7f9f21a26740 1 -- 172.21.15.47:0/455327534 shutdown_connections 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.969+0000 7f9f21a26740 1 -- 172.21.15.47:0/455327534 wait complete. 2024-04-24T22:40:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.970+0000 7f9f21a26740 1 -- start start 2024-04-24T22:40:08.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.971+0000 7f9f21a26740 1 Processor -- start 2024-04-24T22:40:08.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.972+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e654b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.972+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e64f110 0x55e26e646a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.972+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e646fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.972+0000 7f9f21a26740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e26e52bd20 con 0x55e26e650630 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.972+0000 7f9f21a26740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e26e56bdb0 con 0x55e26e64f110 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.972+0000 7f9f21a26740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e26e609290 con 0x55e26e636270 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.972+0000 7f9f21a26740 10 client.0 _ll_register_callbacks cb 0x55e26e566f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.972+0000 7f9f1cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e646fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.972+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e654b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f1cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e646fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57838/0 (socket says 172.21.15.47:57838) 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f1cc00640 1 -- 172.21.15.47:0/3218450962 learned_addr learned my addr 172.21.15.47:0/3218450962 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f17400640 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e64f110 0x55e26e646a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f1cc00640 1 -- 172.21.15.47:0/3218450962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 msgr2=0x55e26e654b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e654b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f1cc00640 1 -- 172.21.15.47:0/3218450962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e64f110 msgr2=0x55e26e646a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e64f110 0x55e26e646a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f1cc00640 1 -- 172.21.15.47:0/3218450962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26e34e6b0 con 0x55e26e650630 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e646fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9f1002a790 tx=0x7f9f10079710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3f31f154d3c775 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f04c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f04c00640 1 -- 172.21.15.47:0/3218450962 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10091020 con 0x55e26e650630 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f04c00640 1 -- 172.21.15.47:0/3218450962 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9f100ab0b0 con 0x55e26e650630 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.973+0000 7f9f21a26740 1 -- 172.21.15.47:0/3218450962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f0801b050 con 0x55e26e650630 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f04c00640 1 -- 172.21.15.47:0/3218450962 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9f10096050 con 0x55e26e650630 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f21a26740 10 client.4542 fetch_fsmap learned FSMap version 17 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f21a26740 1 -- 172.21.15.47:0/3218450962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55e26e647ca0 con 0x55e26e650630 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f04c00640 1 -- 172.21.15.47:0/3218450962 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f0801b050 con 0x55e26e650630 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f04c00640 1 -- 172.21.15.47:0/3218450962 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f9f1004e060 con 0x55e26e650630 2024-04-24T22:40:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f21a26740 10 client.4542 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T22:40:08.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f21a26740 10 client.4542 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T22:40:08.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f21a26740 20 client.4542 populate_metadata read hostname 'smithi047' 2024-04-24T22:40:08.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f21a26740 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.47:6832/2908847255,v1:172.21.15.47:6833/2908847255] conn(0x55e26e66b820 0x55e26e66dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:08.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f21a26740 4 client.4542 mds_command: new command op to 4238 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:40:08.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f21a26740 1 -- 172.21.15.47:0/3218450962 --> [v2:172.21.15.47:6832/2908847255,v1:172.21.15.47:6833/2908847255] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e26e57d330 con 0x55e26e66b820 2024-04-24T22:40:08.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.974+0000 7f9f17e00640 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.47:6832/2908847255,v1:172.21.15.47:6833/2908847255] conn(0x55e26e66b820 0x55e26e66dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:08.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.975+0000 7f9f17e00640 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.47:6832/2908847255,v1:172.21.15.47:6833/2908847255] conn(0x55e26e66b820 0x55e26e66dc10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f91e861a217624d5 server_cookie=430781d3379bcb58 in_seq=0 out_seq=0 2024-04-24T22:40:08.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:08.975+0000 7f9f04c00640 10 client.4542 ms_handle_connect on v2:172.21.15.47:6832/2908847255 2024-04-24T22:40:09.933 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:40:09.940 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:40:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.846+0000 7f9f04c00640 1 -- 172.21.15.47:0/3218450962 <== mds.4 v2:172.21.15.47:6832/2908847255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e26e57d330 con 0x55e26e66b820 2024-04-24T22:40:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.846+0000 7f9f04c00640 10 client.4542 handle_command_reply: tid=0 2024-04-24T22:40:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.846+0000 7f9f21a26740 1 client.4542 shutdown 2024-04-24T22:40:11.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.846+0000 7f9f21a26740 1 -- 172.21.15.47:0/3218450962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 msgr2=0x55e26e646fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.846+0000 7f9f21a26740 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e646fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9f1002a790 tx=0x7f9f10079710 comp rx=0 tx=0).stop 2024-04-24T22:40:11.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.846+0000 7f9f21a26740 1 -- 172.21.15.47:0/3218450962 shutdown_connections 2024-04-24T22:40:11.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.847+0000 7f9f21a26740 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.47:6832/2908847255,v1:172.21.15.47:6833/2908847255] conn(0x55e26e66b820 0x55e26e66dc10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.847+0000 7f9f21a26740 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e646fc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.847+0000 7f9f21a26740 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e64f110 0x55e26e646a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.847+0000 7f9f21a26740 1 --2- 172.21.15.47:0/3218450962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e654b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.847+0000 7f9f04c00640 0 client.4542 ms_handle_reset on v2:172.21.15.47:6832/2908847255 2024-04-24T22:40:11.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.847+0000 7f9f21a26740 1 -- 172.21.15.47:0/3218450962 >> 172.21.15.47:0/3218450962 conn(0x55e26e60b250 msgr2=0x55e26e60d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:11.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.847+0000 7f9f21a26740 1 -- 172.21.15.47:0/3218450962 shutdown_connections 2024-04-24T22:40:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.847+0000 7f9f21a26740 1 -- 172.21.15.47:0/3218450962 wait complete. 2024-04-24T22:40:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.847+0000 7f9f21a26740 20 client.4542 trim_cache size 0 max 16384 2024-04-24T22:40:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.867+0000 7f9f21a26740 1 Processor -- start 2024-04-24T22:40:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.867+0000 7f9f21a26740 1 -- start start 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.868+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e60d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.868+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e64f110 0x55e26e64e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.868+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e64e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.868+0000 7f9f21a26740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1814aa00 con 0x55e26e650630 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.868+0000 7f9f21a26740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e26e651fc0 con 0x55e26e64f110 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.868+0000 7f9f21a26740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e26e650f10 con 0x55e26e636270 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.868+0000 7f9f1cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e64e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.868+0000 7f9f1cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e64e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57850/0 (socket says 172.21.15.47:57850) 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.868+0000 7f9f1cc00640 1 -- 172.21.15.47:0/2241244455 learned_addr learned my addr 172.21.15.47:0/2241244455 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f17e00640 1 --2- 172.21.15.47:0/2241244455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e60d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f17400640 1 --2- 172.21.15.47:0/2241244455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e64f110 0x55e26e64e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f1cc00640 1 -- 172.21.15.47:0/2241244455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 msgr2=0x55e26e60d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/2241244455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e60d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f1cc00640 1 -- 172.21.15.47:0/2241244455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e64f110 msgr2=0x55e26e64e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/2241244455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e64f110 0x55e26e64e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f1cc00640 1 -- 172.21.15.47:0/2241244455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26e64bbf0 con 0x55e26e650630 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/2241244455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e64e690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9f100ab010 tx=0x7f9f10048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b137aeb73882e8de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f07400640 1 -- 172.21.15.47:0/2241244455 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10091030 con 0x55e26e650630 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f07400640 1 -- 172.21.15.47:0/2241244455 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f100776f0 con 0x55e26e650630 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.869+0000 7f9f07400640 1 -- 172.21.15.47:0/2241244455 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10079190 con 0x55e26e650630 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.870+0000 7f9f21a26740 1 -- 172.21.15.47:0/2241244455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 msgr2=0x55e26e64e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.870+0000 7f9f21a26740 1 --2- 172.21.15.47:0/2241244455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e64e690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9f100ab010 tx=0x7f9f10048000 comp rx=0 tx=0).stop 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.870+0000 7f9f21a26740 1 -- 172.21.15.47:0/2241244455 shutdown_connections 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.870+0000 7f9f21a26740 1 --2- 172.21.15.47:0/2241244455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e650630 0x55e26e64e690 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.870+0000 7f9f21a26740 1 --2- 172.21.15.47:0/2241244455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e64f110 0x55e26e64e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.870+0000 7f9f21a26740 1 --2- 172.21.15.47:0/2241244455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e60d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.870+0000 7f9f21a26740 1 -- 172.21.15.47:0/2241244455 >> 172.21.15.47:0/2241244455 conn(0x55e26e60b250 msgr2=0x55e26e60b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.870+0000 7f9f21a26740 1 -- 172.21.15.47:0/2241244455 shutdown_connections 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.870+0000 7f9f21a26740 1 -- 172.21.15.47:0/2241244455 wait complete. 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.871+0000 7f9f21a26740 1 -- start start 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.871+0000 7f9f21a26740 1 Processor -- start 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e671de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e64f110 0x55e26e64fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f21a26740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e650630 0x55e26e6585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f21a26740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e26e661ec0 con 0x55e26e64f110 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f21a26740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e26e64ed30 con 0x55e26e650630 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f21a26740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e26e662530 con 0x55e26e636270 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f21a26740 10 client.0 _ll_register_callbacks cb 0x7f9f10009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:40:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f17400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e64f110 0x55e26e64fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f17400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e64f110 0x55e26e64fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57854/0 (socket says 172.21.15.47:57854) 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f17400640 1 -- 172.21.15.47:0/2681926606 learned_addr learned my addr 172.21.15.47:0/2681926606 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f17e00640 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e671de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.872+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e650630 0x55e26e6585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f17400640 1 -- 172.21.15.47:0/2681926606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 msgr2=0x55e26e671de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f17400640 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e671de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f17400640 1 -- 172.21.15.47:0/2681926606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e650630 msgr2=0x55e26e6585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f17400640 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e650630 0x55e26e6585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f17400640 1 -- 172.21.15.47:0/2681926606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26e6c9c50 con 0x55e26e64f110 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f1cc00640 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e650630 0x55e26e6585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f17e00640 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e671de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f17400640 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e64f110 0x55e26e64fbb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9f18151570 tx=0x7f9f00015780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f966181b56200d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f04c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-24T22:40:11.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f04c00640 1 -- 172.21.15.47:0/2681926606 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00003b80 con 0x55e26e64f110 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f04c00640 1 -- 172.21.15.47:0/2681926606 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9f00003d20 con 0x55e26e64f110 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.873+0000 7f9f21a26740 1 -- 172.21.15.47:0/2681926606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f100776f0 con 0x55e26e64f110 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f04c00640 1 -- 172.21.15.47:0/2681926606 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9f00005110 con 0x55e26e64f110 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f21a26740 10 client.4548 fetch_fsmap learned FSMap version 17 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f21a26740 1 -- 172.21.15.47:0/2681926606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55e26e6c98e0 con 0x55e26e64f110 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f04c00640 1 -- 172.21.15.47:0/2681926606 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f100776f0 con 0x55e26e64f110 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f04c00640 1 -- 172.21.15.47:0/2681926606 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f9f000052b0 con 0x55e26e64f110 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f21a26740 10 client.4548 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f21a26740 10 client.4548 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f21a26740 20 client.4548 populate_metadata read hostname 'smithi047' 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f21a26740 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.47:6832/2908847255,v1:172.21.15.47:6833/2908847255] conn(0x55e26e6ccb80 0x55e26e6bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f21a26740 4 client.4548 mds_command: new command op to 4238 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f21a26740 1 -- 172.21.15.47:0/2681926606 --> [v2:172.21.15.47:6832/2908847255,v1:172.21.15.47:6833/2908847255] -- command(tid 0: {"prefix": "status"}) -- 0x55e26e577680 con 0x55e26e6ccb80 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.874+0000 7f9f17e00640 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.47:6832/2908847255,v1:172.21.15.47:6833/2908847255] conn(0x55e26e6ccb80 0x55e26e6bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:11.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f17e00640 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.47:6832/2908847255,v1:172.21.15.47:6833/2908847255] conn(0x55e26e6ccb80 0x55e26e6bf7a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8736c8689e65bdf2 server_cookie=fe80fa24914d14e4 in_seq=0 out_seq=0 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f04c00640 10 client.4548 ms_handle_connect on v2:172.21.15.47:6832/2908847255 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f04c00640 1 -- 172.21.15.47:0/2681926606 <== mds.4 v2:172.21.15.47:6832/2908847255 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e26e577680 con 0x55e26e6ccb80 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f04c00640 10 client.4548 handle_command_reply: tid=0 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 client.4548 shutdown 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 -- 172.21.15.47:0/2681926606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e64f110 msgr2=0x55e26e64fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e64f110 0x55e26e64fbb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9f18151570 tx=0x7f9f00015780 comp rx=0 tx=0).stop 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 -- 172.21.15.47:0/2681926606 shutdown_connections 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.47:6832/2908847255,v1:172.21.15.47:6833/2908847255] conn(0x55e26e6ccb80 0x55e26e6bf7a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e26e650630 0x55e26e6585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e26e64f110 0x55e26e64fbb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 --2- 172.21.15.47:0/2681926606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e26e636270 0x55e26e671de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 -- 172.21.15.47:0/2681926606 >> 172.21.15.47:0/2681926606 conn(0x55e26e60b250 msgr2=0x55e26e651260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f04c00640 0 client.4548 ms_handle_reset on v2:172.21.15.47:6832/2908847255 2024-04-24T22:40:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 -- 172.21.15.47:0/2681926606 shutdown_connections 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.875+0000 7f9f21a26740 1 -- 172.21.15.47:0/2681926606 wait complete. 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f21a26740 20 client.4548 trim_cache size 0 max 16384 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 4, 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4238, 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 11.045939747, 2024-04-24T22:40:11.885 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 17, 2024-04-24T22:40:11.886 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 21, 2024-04-24T22:40:11.886 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 21, 2024-04-24T22:40:11.886 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 42.200833465000002 2024-04-24T22:40:11.886 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:40:11.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 -- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9ef406be30 msgr2=0x7f9ef406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9ef406be30 0x7f9ef406e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9f080011e0 tx=0x7f9f08007d30 comp rx=0 tx=0).stop 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 -- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f18154560 msgr2=0x7f9f18151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f18154560 0x7f9f18151ce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f10031000 tx=0x7f9f100046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 -- 172.21.15.47:0/4070460248 shutdown_connections 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9ef406be30 0x7f9ef406e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f18154560 0x7f9f18151ce0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f180cbb40 0x7f9f180b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 --2- 172.21.15.47:0/4070460248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f18073280 0x7f9f181f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 -- 172.21.15.47:0/4070460248 >> 172.21.15.47:0/4070460248 conn(0x7f9f180672c0 msgr2=0x7f9f180cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 -- 172.21.15.47:0/4070460248 shutdown_connections 2024-04-24T22:40:11.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:11.876+0000 7f9f1d600640 1 -- 172.21.15.47:0/4070460248 wait complete. 2024-04-24T22:40:11.897 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-24T22:40:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.090+0000 7f6e1ac00640 1 Processor -- start 2024-04-24T22:40:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.090+0000 7f6e1ac00640 1 -- start start 2024-04-24T22:40:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.092+0000 7f6e1ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14073040 0x7f6e141411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.092+0000 7f6e1ac00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6e1406bb90 0x7f6e14141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.092+0000 7f6e1ac00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e1415c3d0 con 0x7f6e14072c60 2024-04-24T22:40:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.092+0000 7f6e1ac00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6e14158bf0 con 0x7f6e1406bb90 2024-04-24T22:40:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.092+0000 7f6e1ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6e1415c6b0 con 0x7f6e14073040 2024-04-24T22:40:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.092+0000 7f6e1a200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6e1406bb90 0x7f6e14141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.093+0000 7f6e18e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14073040 0x7f6e141411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.093+0000 7f6e19800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6e14072c60 0x7f6e140c2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:54264/0 (socket says 172.21.15.47:54264) 2024-04-24T22:40:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.093+0000 7f6e19800640 1 -- 172.21.15.47:0/3314911187 learned_addr learned my addr 172.21.15.47:0/3314911187 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.093+0000 7f6e1a200640 1 -- 172.21.15.47:0/3314911187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14073040 msgr2=0x7f6e141411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.093+0000 7f6e1a200640 1 --2- 172.21.15.47:0/3314911187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14073040 0x7f6e141411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.093+0000 7f6e1a200640 1 -- 172.21.15.47:0/3314911187 >> v1:172.21.15.113:6789/0 conn(0x7f6e14072c60 legacy=0x7f6e140c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.093+0000 7f6e1a200640 1 -- 172.21.15.47:0/3314911187 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e141585a0 con 0x7f6e1406bb90 2024-04-24T22:40:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.094+0000 7f6e1a200640 1 --2- 172.21.15.47:0/3314911187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e1406bb90 0x7f6e14141730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0097a0 tx=0x7f6e0c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=631057707f7985e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.094+0000 7f6e13e00640 1 -- 172.21.15.47:0/3314911187 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c037070 con 0x7f6e1406bb90 2024-04-24T22:40:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.094+0000 7f6e13e00640 1 -- 172.21.15.47:0/3314911187 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0c032440 con 0x7f6e1406bb90 2024-04-24T22:40:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.094+0000 7f6e13e00640 1 -- 172.21.15.47:0/3314911187 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c032810 con 0x7f6e1406bb90 2024-04-24T22:40:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.095+0000 7f6e1ac00640 1 -- 172.21.15.47:0/3314911187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e1406bb90 msgr2=0x7f6e14141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.095+0000 7f6e1ac00640 1 --2- 172.21.15.47:0/3314911187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e1406bb90 0x7f6e14141730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0097a0 tx=0x7f6e0c02a210 comp rx=0 tx=0).stop 2024-04-24T22:40:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.095+0000 7f6e1ac00640 1 -- 172.21.15.47:0/3314911187 shutdown_connections 2024-04-24T22:40:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.095+0000 7f6e1ac00640 1 --2- 172.21.15.47:0/3314911187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e1406bb90 0x7f6e14141730 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.095+0000 7f6e1ac00640 1 --2- 172.21.15.47:0/3314911187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14073040 0x7f6e141411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.095+0000 7f6e1ac00640 1 -- 172.21.15.47:0/3314911187 >> 172.21.15.47:0/3314911187 conn(0x7f6e140cb900 msgr2=0x7f6e140cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.095+0000 7f6e1ac00640 1 -- 172.21.15.47:0/3314911187 shutdown_connections 2024-04-24T22:40:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.095+0000 7f6e1ac00640 1 -- 172.21.15.47:0/3314911187 wait complete. 2024-04-24T22:40:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.097+0000 7f6e1ac00640 1 Processor -- start 2024-04-24T22:40:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.097+0000 7f6e1ac00640 1 -- start start 2024-04-24T22:40:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e1ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e1406bb90 0x7f6e140c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e1ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14072c60 0x7f6e141f0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e1ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e14073040 0x7f6e141545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e1ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14159080 con 0x7f6e1406bb90 2024-04-24T22:40:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e1ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14158270 con 0x7f6e14073040 2024-04-24T22:40:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e1ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6e14159200 con 0x7f6e14072c60 2024-04-24T22:40:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e19800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e1406bb90 0x7f6e140c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e18e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14072c60 0x7f6e141f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e18e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14072c60 0x7f6e141f0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:44056/0 (socket says 172.21.15.47:44056) 2024-04-24T22:40:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e18e00640 1 -- 172.21.15.47:0/503024595 learned_addr learned my addr 172.21.15.47:0/503024595 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.098+0000 7f6e1a200640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e14073040 0x7f6e141545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.099+0000 7f6e18e00640 1 -- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e14073040 msgr2=0x7f6e141545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.099+0000 7f6e18e00640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e14073040 0x7f6e141545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.099+0000 7f6e18e00640 1 -- 172.21.15.47:0/503024595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e1406bb90 msgr2=0x7f6e140c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.099+0000 7f6e18e00640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e1406bb90 0x7f6e140c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.099+0000 7f6e18e00640 1 -- 172.21.15.47:0/503024595 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e14155320 con 0x7f6e14072c60 2024-04-24T22:40:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.099+0000 7f6e19800640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e1406bb90 0x7f6e140c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.099+0000 7f6e18e00640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14072c60 0x7f6e141f0900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000e8a0 tx=0x7f6e0000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.099+0000 7f6e11600640 1 -- 172.21.15.47:0/503024595 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00017070 con 0x7f6e14072c60 2024-04-24T22:40:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.100+0000 7f6e11600640 1 -- 172.21.15.47:0/503024595 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0000c940 con 0x7f6e14072c60 2024-04-24T22:40:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.100+0000 7f6e1ac00640 1 -- 172.21.15.47:0/503024595 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e14153af0 con 0x7f6e14072c60 2024-04-24T22:40:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.100+0000 7f6e11600640 1 -- 172.21.15.47:0/503024595 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0000cd30 con 0x7f6e14072c60 2024-04-24T22:40:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.100+0000 7f6e1ac00640 1 -- 172.21.15.47:0/503024595 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e141540b0 con 0x7f6e14072c60 2024-04-24T22:40:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.102+0000 7f6e11600640 1 -- 172.21.15.47:0/503024595 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e0001a070 con 0x7f6e14072c60 2024-04-24T22:40:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.103+0000 7f6e11600640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6df406be30 0x7f6df406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.103+0000 7f6e11600640 1 -- 172.21.15.47:0/503024595 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6e0008afd0 con 0x7f6e14072c60 2024-04-24T22:40:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.103+0000 7f6e19800640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6df406be30 0x7f6df406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.103+0000 7f6e1f081740 1 Processor -- start 2024-04-24T22:40:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.104+0000 7f6e1f081740 1 -- start start 2024-04-24T22:40:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.104+0000 7f6e19800640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6df406be30 0x7f6df406e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6e140ca8f0 tx=0x7f6e08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f577ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5731e00 0x55d5f5733d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f5734250 0x55d5f575f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e1f081740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d5f5635ed0 con 0x55d5f5731e00 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e1f081740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d5f5698df0 con 0x55d5f5734250 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e1f081740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55d5f56418d0 con 0x55d5f575f270 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e18e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5731e00 0x55d5f5733d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e19800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f577ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e1a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f5734250 0x55d5f575f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e19800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f577ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:44072/0 (socket says 172.21.15.47:44072) 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e18e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5731e00 0x55d5f5733d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57868/0 (socket says 172.21.15.47:57868) 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e19800640 1 -- 172.21.15.47:0/3496825942 learned_addr learned my addr 172.21.15.47:0/3496825942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e18e00640 1 -- 172.21.15.47:0/3496825942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 msgr2=0x55d5f577ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e18e00640 1 --2- 172.21.15.47:0/3496825942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f577ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e18e00640 1 -- 172.21.15.47:0/3496825942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f5734250 msgr2=0x55d5f575f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e18e00640 1 --2- 172.21.15.47:0/3496825942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f5734250 0x55d5f575f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.105+0000 7f6e18e00640 1 -- 172.21.15.47:0/3496825942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f53ad2d0 con 0x55d5f5731e00 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.106+0000 7f6e18e00640 1 --2- 172.21.15.47:0/3496825942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5731e00 0x55d5f5733d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e000894c0 tx=0x7f6e00058c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7905a37ee0023f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.106+0000 7f6e06000640 1 -- 172.21.15.47:0/3496825942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00021040 con 0x55d5f5731e00 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.106+0000 7f6e06000640 1 -- 172.21.15.47:0/3496825942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e00082e50 con 0x55d5f5731e00 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.106+0000 7f6e06000640 1 -- 172.21.15.47:0/3496825942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00061e50 con 0x55d5f5731e00 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.106+0000 7f6e1f081740 1 -- 172.21.15.47:0/3496825942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5731e00 msgr2=0x55d5f5733d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.106+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3496825942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5731e00 0x55d5f5733d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e000894c0 tx=0x7f6e00058c80 comp rx=0 tx=0).stop 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.107+0000 7f6e1f081740 1 -- 172.21.15.47:0/3496825942 shutdown_connections 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.107+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3496825942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f5734250 0x55d5f575f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.107+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3496825942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5731e00 0x55d5f5733d10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.107+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3496825942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f577ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.107+0000 7f6e1f081740 1 -- 172.21.15.47:0/3496825942 >> 172.21.15.47:0/3496825942 conn(0x55d5f5732c90 msgr2=0x55d5f577a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.107+0000 7f6e1f081740 1 -- 172.21.15.47:0/3496825942 shutdown_connections 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.107+0000 7f6e1f081740 1 -- 172.21.15.47:0/3496825942 wait complete. 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.107+0000 7f6e1f081740 1 -- start start 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.108+0000 7f6e1f081740 1 Processor -- start 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f5736400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f57614c0 0x55d5f578a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f575c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1f081740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d5f5654d20 con 0x55d5f5770060 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1f081740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d5f5694db0 con 0x55d5f57614c0 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1f081740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55d5f5732290 con 0x55d5f575f270 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1f081740 10 client.0 _ll_register_callbacks cb 0x55d5f568ff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f575c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f575c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57870/0 (socket says 172.21.15.47:57870) 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1a200640 1 -- 172.21.15.47:0/2691230868 learned_addr learned my addr 172.21.15.47:0/2691230868 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e19800640 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f5736400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e18e00640 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f57614c0 0x55d5f578a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1a200640 1 -- 172.21.15.47:0/2691230868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 msgr2=0x55d5f5736400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1a200640 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f5736400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1a200640 1 -- 172.21.15.47:0/2691230868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f57614c0 msgr2=0x55d5f578a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1a200640 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f57614c0 0x55d5f578a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.109+0000 7f6e1a200640 1 -- 172.21.15.47:0/2691230868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5f54776b0 con 0x55d5f5770060 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.110+0000 7f6e1a200640 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f575c970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0097a0 tx=0x7f6e0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232d57375ebd0247 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.110+0000 7f6dfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.110+0000 7f6dfaa00640 1 -- 172.21.15.47:0/2691230868 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c037040 con 0x55d5f5770060 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.110+0000 7f6dfaa00640 1 -- 172.21.15.47:0/2691230868 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6e0c032cc0 con 0x55d5f5770060 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.110+0000 7f6e1f081740 1 -- 172.21.15.47:0/2691230868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e00082e50 con 0x55d5f5770060 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.110+0000 7f6dfaa00640 1 -- 172.21.15.47:0/2691230868 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f6e0c006e10 con 0x55d5f5770060 2024-04-24T22:40:12.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.110+0000 7f6e1f081740 10 client.4557 fetch_fsmap learned FSMap version 17 2024-04-24T22:40:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.110+0000 7f6e1f081740 1 -- 172.21.15.47:0/2691230868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d5f57879c0 con 0x55d5f5770060 2024-04-24T22:40:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.110+0000 7f6dfaa00640 1 -- 172.21.15.47:0/2691230868 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e00082e50 con 0x55d5f5770060 2024-04-24T22:40:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.111+0000 7f6dfaa00640 1 -- 172.21.15.47:0/2691230868 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f6e0c04d050 con 0x55d5f5770060 2024-04-24T22:40:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.111+0000 7f6e1f081740 10 client.4557 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T22:40:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.111+0000 7f6e1f081740 10 client.4557 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-24T22:40:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.111+0000 7f6e1f081740 20 client.4557 populate_metadata read hostname 'smithi047' 2024-04-24T22:40:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.111+0000 7f6e1f081740 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.47:6836/2173854280,v1:172.21.15.47:6837/2173854280] conn(0x55d5f5779f70 0x55d5f5796890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.111+0000 7f6e1f081740 4 client.4557 mds_command: new command op to 4241 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.111+0000 7f6e1f081740 1 -- 172.21.15.47:0/2691230868 --> [v2:172.21.15.47:6836/2173854280,v1:172.21.15.47:6837/2173854280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5f56a6330 con 0x55d5f5779f70 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.111+0000 7f6e19800640 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.47:6836/2173854280,v1:172.21.15.47:6837/2173854280] conn(0x55d5f5779f70 0x55d5f5796890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.112+0000 7f6e19800640 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.47:6836/2173854280,v1:172.21.15.47:6837/2173854280] conn(0x55d5f5779f70 0x55d5f5796890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d9f96ec1a5ec108 server_cookie=c40d1bbee0ffda27 in_seq=0 out_seq=0 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.112+0000 7f6dfaa00640 10 client.4557 ms_handle_connect on v2:172.21.15.47:6836/2173854280 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.112+0000 7f6dfaa00640 1 -- 172.21.15.47:0/2691230868 <== mds.3 v2:172.21.15.47:6836/2173854280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5f56a6330 con 0x55d5f5779f70 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6dfaa00640 10 client.4557 handle_command_reply: tid=0 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 client.4557 shutdown 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 -- 172.21.15.47:0/2691230868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 msgr2=0x55d5f575c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f575c970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0097a0 tx=0x7f6e0c004730 comp rx=0 tx=0).stop 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 -- 172.21.15.47:0/2691230868 shutdown_connections 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.47:6836/2173854280,v1:172.21.15.47:6837/2173854280] conn(0x55d5f5779f70 0x55d5f5796890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f575c970 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f57614c0 0x55d5f578a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 --2- 172.21.15.47:0/2691230868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f5736400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6dfaa00640 0 client.4557 ms_handle_reset on v2:172.21.15.47:6836/2173854280 2024-04-24T22:40:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 -- 172.21.15.47:0/2691230868 >> 172.21.15.47:0/2691230868 conn(0x55d5f5734250 msgr2=0x55d5f57769f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 -- 172.21.15.47:0/2691230868 shutdown_connections 2024-04-24T22:40:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 1 -- 172.21.15.47:0/2691230868 wait complete. 2024-04-24T22:40:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.113+0000 7f6e1f081740 20 client.4557 trim_cache size 0 max 16384 2024-04-24T22:40:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.133+0000 7f6e1f081740 1 Processor -- start 2024-04-24T22:40:12.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.133+0000 7f6e1f081740 1 -- start start 2024-04-24T22:40:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f575f270 0x55d5f57914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f57614c0 0x55d5f575bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f5770060 0x55d5f575c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1f081740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141f3700 con 0x55d5f57614c0 2024-04-24T22:40:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1f081740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d5f5787ca0 con 0x55d5f575f270 2024-04-24T22:40:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1f081740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55d5f575d3d0 con 0x55d5f5770060 2024-04-24T22:40:12.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e18e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f57614c0 0x55d5f575bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e18e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f57614c0 0x55d5f575bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57880/0 (socket says 172.21.15.47:57880) 2024-04-24T22:40:12.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1a200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f5770060 0x55d5f575c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e18e00640 1 -- 172.21.15.47:0/3398880366 learned_addr learned my addr 172.21.15.47:0/3398880366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e19800640 1 --2- 172.21.15.47:0/3398880366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f575f270 0x55d5f57914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1a200640 1 -- 172.21.15.47:0/3398880366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f575f270 msgr2=0x55d5f57914c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1a200640 1 --2- 172.21.15.47:0/3398880366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f575f270 0x55d5f57914c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1a200640 1 -- 172.21.15.47:0/3398880366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f57614c0 msgr2=0x55d5f575bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1a200640 1 --2- 172.21.15.47:0/3398880366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f57614c0 0x55d5f575bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e1a200640 1 -- 172.21.15.47:0/3398880366 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f577aca0 con 0x55d5f5770060 2024-04-24T22:40:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e18e00640 1 --2- 172.21.15.47:0/3398880366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f57614c0 0x55d5f575bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.134+0000 7f6e19800640 1 --2- 172.21.15.47:0/3398880366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f575f270 0x55d5f57914c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e1a200640 1 --2- 172.21.15.47:0/3398880366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f5770060 0x55d5f575c1b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006a90 tx=0x7f6e0c03e070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab5a10e09ee1877 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e05600640 1 -- 172.21.15.47:0/3398880366 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c037040 con 0x55d5f5770060 2024-04-24T22:40:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e05600640 1 -- 172.21.15.47:0/3398880366 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0c004340 con 0x55d5f5770060 2024-04-24T22:40:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e05600640 1 -- 172.21.15.47:0/3398880366 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c032a00 con 0x55d5f5770060 2024-04-24T22:40:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e1f081740 1 -- 172.21.15.47:0/3398880366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f5770060 msgr2=0x55d5f575c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3398880366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f5770060 0x55d5f575c1b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006a90 tx=0x7f6e0c03e070 comp rx=0 tx=0).stop 2024-04-24T22:40:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e1f081740 1 -- 172.21.15.47:0/3398880366 shutdown_connections 2024-04-24T22:40:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3398880366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f5770060 0x55d5f575c1b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3398880366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f57614c0 0x55d5f575bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3398880366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f575f270 0x55d5f57914c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e1f081740 1 -- 172.21.15.47:0/3398880366 >> 172.21.15.47:0/3398880366 conn(0x55d5f5734250 msgr2=0x55d5f5734880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.135+0000 7f6e1f081740 1 -- 172.21.15.47:0/3398880366 shutdown_connections 2024-04-24T22:40:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.136+0000 7f6e1f081740 1 -- 172.21.15.47:0/3398880366 wait complete. 2024-04-24T22:40:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.136+0000 7f6e1f081740 1 -- start start 2024-04-24T22:40:12.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.136+0000 7f6e1f081740 1 Processor -- start 2024-04-24T22:40:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.137+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f5762160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.137+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f57614c0 0x55d5f572fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.137+0000 7f6e1f081740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f5777ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.137+0000 7f6e1f081740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d5f577a440 con 0x55d5f5770060 2024-04-24T22:40:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.137+0000 7f6e1f081740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d5f5779df0 con 0x55d5f57614c0 2024-04-24T22:40:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.137+0000 7f6e1f081740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55d5f577aad0 con 0x55d5f575f270 2024-04-24T22:40:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.137+0000 7f6e1f081740 10 client.0 _ll_register_callbacks cb 0x7f6e0c009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e1a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f5777ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e19800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f5762160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e1a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f5777ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57882/0 (socket says 172.21.15.47:57882) 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e1a200640 1 -- 172.21.15.47:0/3789374934 learned_addr learned my addr 172.21.15.47:0/3789374934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e18e00640 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f57614c0 0x55d5f572fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e1a200640 1 -- 172.21.15.47:0/3789374934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 msgr2=0x55d5f5762160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e1a200640 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f5762160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e1a200640 1 -- 172.21.15.47:0/3789374934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f57614c0 msgr2=0x55d5f572fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e1a200640 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f57614c0 0x55d5f572fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e1a200640 1 -- 172.21.15.47:0/3789374934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5f5778400 con 0x55d5f5770060 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e19800640 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f5762160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6e1a200640 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f5777ec0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006dc0 tx=0x7f6e0c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe49b0f936f1987 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6dfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-24T22:40:12.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6dfaa00640 1 -- 172.21.15.47:0/3789374934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c037070 con 0x55d5f5770060 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.138+0000 7f6dfaa00640 1 -- 172.21.15.47:0/3789374934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6e0c0327e0 con 0x55d5f5770060 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6e1f081740 1 -- 172.21.15.47:0/3789374934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e0c004340 con 0x55d5f5770060 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6dfaa00640 1 -- 172.21.15.47:0/3789374934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f6e0c063050 con 0x55d5f5770060 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6e1f081740 10 client.4563 fetch_fsmap learned FSMap version 17 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6e1f081740 1 -- 172.21.15.47:0/3789374934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d5f579bee0 con 0x55d5f5770060 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6dfaa00640 1 -- 172.21.15.47:0/3789374934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0c004340 con 0x55d5f5770060 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6dfaa00640 1 -- 172.21.15.47:0/3789374934 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f6e0c04d070 con 0x55d5f5770060 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6e1f081740 10 client.4563 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6e1f081740 10 client.4563 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6e1f081740 20 client.4563 populate_metadata read hostname 'smithi047' 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.47:6836/2173854280,v1:172.21.15.47:6837/2173854280] conn(0x55d5f5784600 0x55d5f57fe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6e1f081740 4 client.4563 mds_command: new command op to 4241 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:40:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6e1f081740 1 -- 172.21.15.47:0/3789374934 --> [v2:172.21.15.47:6836/2173854280,v1:172.21.15.47:6837/2173854280] -- command(tid 0: {"prefix": "status"}) -- 0x55d5f56a0680 con 0x55d5f5784600 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.139+0000 7f6e19800640 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.47:6836/2173854280,v1:172.21.15.47:6837/2173854280] conn(0x55d5f5784600 0x55d5f57fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.140+0000 7f6e19800640 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.47:6836/2173854280,v1:172.21.15.47:6837/2173854280] conn(0x55d5f5784600 0x55d5f57fe510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ec4ea3aa8ca5475 server_cookie=7aa78b033092bf92 in_seq=0 out_seq=0 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.140+0000 7f6dfaa00640 10 client.4563 ms_handle_connect on v2:172.21.15.47:6836/2173854280 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.140+0000 7f6dfaa00640 1 -- 172.21.15.47:0/3789374934 <== mds.3 v2:172.21.15.47:6836/2173854280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55d5f56a0680 con 0x55d5f5784600 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.140+0000 7f6dfaa00640 10 client.4563 handle_command_reply: tid=0 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.140+0000 7f6e1f081740 1 client.4563 shutdown 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 1 -- 172.21.15.47:0/3789374934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 msgr2=0x55d5f5777ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f5777ec0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006dc0 tx=0x7f6e0c009190 comp rx=0 tx=0).stop 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 1 -- 172.21.15.47:0/3789374934 shutdown_connections 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.47:6836/2173854280,v1:172.21.15.47:6837/2173854280] conn(0x55d5f5784600 0x55d5f57fe510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5f5770060 0x55d5f5777ec0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5f57614c0 0x55d5f572fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 1 --2- 172.21.15.47:0/3789374934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d5f575f270 0x55d5f5762160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 1 -- 172.21.15.47:0/3789374934 >> 172.21.15.47:0/3789374934 conn(0x55d5f5734250 msgr2=0x55d5f572bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6dfaa00640 0 client.4563 ms_handle_reset on v2:172.21.15.47:6836/2173854280 2024-04-24T22:40:12.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 1 -- 172.21.15.47:0/3789374934 shutdown_connections 2024-04-24T22:40:12.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 1 -- 172.21.15.47:0/3789374934 wait complete. 2024-04-24T22:40:12.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.141+0000 7f6e1f081740 20 client.4563 trim_cache size 0 max 16384 2024-04-24T22:40:12.150 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:40:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:40:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 3, 2024-04-24T22:40:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4241, 2024-04-24T22:40:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-24T22:40:12.150 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-24T22:40:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-24T22:40:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 17.337925610999999, 2024-04-24T22:40:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 17, 2024-04-24T22:40:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 21, 2024-04-24T22:40:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 21, 2024-04-24T22:40:12.151 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 42.455527984 2024-04-24T22:40:12.151 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:40:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 -- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6df406be30 msgr2=0x7f6df406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6df406be30 0x7f6df406e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6e140ca8f0 tx=0x7f6e08009040 comp rx=0 tx=0).stop 2024-04-24T22:40:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 -- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14072c60 msgr2=0x7f6e141f0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14072c60 0x7f6e141f0900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000e8a0 tx=0x7f6e0000ed60 comp rx=0 tx=0).stop 2024-04-24T22:40:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 -- 172.21.15.47:0/503024595 shutdown_connections 2024-04-24T22:40:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6df406be30 0x7f6df406e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e14073040 0x7f6e141545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e14072c60 0x7f6e141f0900 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 --2- 172.21.15.47:0/503024595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e1406bb90 0x7f6e140c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 -- 172.21.15.47:0/503024595 >> 172.21.15.47:0/503024595 conn(0x7f6e140cb900 msgr2=0x7f6e141578b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 -- 172.21.15.47:0/503024595 shutdown_connections 2024-04-24T22:40:12.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.142+0000 7f6e1ac00640 1 -- 172.21.15.47:0/503024595 wait complete. 2024-04-24T22:40:12.163 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-24T22:40:12.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.368+0000 7f0876800640 1 Processor -- start 2024-04-24T22:40:12.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.369+0000 7f0876800640 1 -- start start 2024-04-24T22:40:12.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.370+0000 7f0876800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f087006b3d0 0x7f0870158220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.370+0000 7f0876800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0870158760 0x7f08700c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.370+0000 7f0876800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f087015c750 con 0x7f0870152670 2024-04-24T22:40:12.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.370+0000 7f0876800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f087006b930 con 0x7f087006b3d0 2024-04-24T22:40:12.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.370+0000 7f0876800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f087015b990 con 0x7f0870158760 2024-04-24T22:40:12.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.370+0000 7f0875400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0870152670 0x7f087006bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51968/0 (socket says 172.21.15.47:51968) 2024-04-24T22:40:12.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.370+0000 7f0875400640 1 -- 172.21.15.47:0/914685230 learned_addr learned my addr 172.21.15.47:0/914685230 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:12.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.371+0000 7f0875e00640 1 --2- 172.21.15.47:0/914685230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0870158760 0x7f08700c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.371+0000 7f0874a00640 1 --2- 172.21.15.47:0/914685230 >> v2:172.21.15.113:3300/0 conn(0x7f087006b3d0 0x7f0870158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.371+0000 7f086fe00640 1 -- 172.21.15.47:0/914685230 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4204157134 0 0) 0x7f087015c750 con 0x7f0870152670 2024-04-24T22:40:12.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.371+0000 7f086fe00640 1 -- 172.21.15.47:0/914685230 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0854003520 con 0x7f0870152670 2024-04-24T22:40:12.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.371+0000 7f0875e00640 1 -- 172.21.15.47:0/914685230 >> v2:172.21.15.113:3300/0 conn(0x7f087006b3d0 msgr2=0x7f0870158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.371+0000 7f0875e00640 1 --2- 172.21.15.47:0/914685230 >> v2:172.21.15.113:3300/0 conn(0x7f087006b3d0 0x7f0870158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.371+0000 7f0875e00640 1 -- 172.21.15.47:0/914685230 >> v1:172.21.15.47:6789/0 conn(0x7f0870152670 legacy=0x7f087006bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.371+0000 7f0875e00640 1 -- 172.21.15.47:0/914685230 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087015bc70 con 0x7f0870158760 2024-04-24T22:40:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.371+0000 7f0874a00640 1 --2- 172.21.15.47:0/914685230 >> v2:172.21.15.113:3300/0 conn(0x7f087006b3d0 0x7f0870158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.372+0000 7f0875e00640 1 --2- 172.21.15.47:0/914685230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0870158760 0x7f08700c5fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f08680097a0 tx=0x7f086802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f25119548b6b5a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.372+0000 7f086fe00640 1 -- 172.21.15.47:0/914685230 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868037070 con 0x7f0870158760 2024-04-24T22:40:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.372+0000 7f086fe00640 1 -- 172.21.15.47:0/914685230 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0868032440 con 0x7f0870158760 2024-04-24T22:40:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.372+0000 7f086fe00640 1 -- 172.21.15.47:0/914685230 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868032810 con 0x7f0870158760 2024-04-24T22:40:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.373+0000 7f0876800640 1 -- 172.21.15.47:0/914685230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0870158760 msgr2=0x7f08700c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.373+0000 7f0876800640 1 --2- 172.21.15.47:0/914685230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0870158760 0x7f08700c5fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f08680097a0 tx=0x7f086802a210 comp rx=0 tx=0).stop 2024-04-24T22:40:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.373+0000 7f0876800640 1 -- 172.21.15.47:0/914685230 shutdown_connections 2024-04-24T22:40:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.373+0000 7f0876800640 1 --2- 172.21.15.47:0/914685230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0870158760 0x7f08700c5fa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.373+0000 7f0876800640 1 --2- 172.21.15.47:0/914685230 >> v2:172.21.15.113:3300/0 conn(0x7f087006b3d0 0x7f0870158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.373+0000 7f0876800640 1 -- 172.21.15.47:0/914685230 >> 172.21.15.47:0/914685230 conn(0x7f08700c9410 msgr2=0x7f08700c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.373+0000 7f0876800640 1 -- 172.21.15.47:0/914685230 shutdown_connections 2024-04-24T22:40:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.373+0000 7f0876800640 1 -- 172.21.15.47:0/914685230 wait complete. 2024-04-24T22:40:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.375+0000 7f0876800640 1 Processor -- start 2024-04-24T22:40:12.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.375+0000 7f0876800640 1 -- start start 2024-04-24T22:40:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0876800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087006b3d0 0x7f0870144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0876800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0870152670 0x7f0870159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0876800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0870158760 0x7f0870150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0876800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f087006c660 con 0x7f0870152670 2024-04-24T22:40:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0876800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f087015c230 con 0x7f0870158760 2024-04-24T22:40:12.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0876800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f08700c9190 con 0x7f087006b3d0 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0874a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0870152670 0x7f0870159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0874a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0870152670 0x7f0870159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57886/0 (socket says 172.21.15.47:57886) 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0874a00640 1 -- 172.21.15.47:0/784121121 learned_addr learned my addr 172.21.15.47:0/784121121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0875400640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087006b3d0 0x7f0870144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0875e00640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0870158760 0x7f0870150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0874a00640 1 -- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087006b3d0 msgr2=0x7f0870144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.376+0000 7f0874a00640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087006b3d0 0x7f0870144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.377+0000 7f0874a00640 1 -- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0870158760 msgr2=0x7f0870150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.377+0000 7f0874a00640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0870158760 0x7f0870150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.377+0000 7f0874a00640 1 -- 172.21.15.47:0/784121121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087015ab80 con 0x7f0870152670 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.377+0000 7f0875e00640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0870158760 0x7f0870150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.377+0000 7f0874a00640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0870152670 0x7f0870159d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f085c00ebe0 tx=0x7f085c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.377+0000 7f086d600640 1 -- 172.21.15.47:0/784121121 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c017070 con 0x7f0870152670 2024-04-24T22:40:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.377+0000 7f086d600640 1 -- 172.21.15.47:0/784121121 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c00cc30 con 0x7f0870152670 2024-04-24T22:40:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.377+0000 7f086d600640 1 -- 172.21.15.47:0/784121121 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c013470 con 0x7f0870152670 2024-04-24T22:40:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.377+0000 7f0876800640 1 -- 172.21.15.47:0/784121121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08700cc370 con 0x7f0870152670 2024-04-24T22:40:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.377+0000 7f0876800640 1 -- 172.21.15.47:0/784121121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087015a5c0 con 0x7f0870152670 2024-04-24T22:40:12.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.379+0000 7f086d600640 1 -- 172.21.15.47:0/784121121 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f085c01a070 con 0x7f0870152670 2024-04-24T22:40:12.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.381+0000 7f086d600640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f085006be30 0x7f085006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.381+0000 7f086d600640 1 -- 172.21.15.47:0/784121121 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f085c08adb0 con 0x7f0870152670 2024-04-24T22:40:12.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.381+0000 7f0875400640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f085006be30 0x7f085006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.382+0000 7f087abcc740 1 Processor -- start 2024-04-24T22:40:12.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.382+0000 7f087abcc740 1 -- start start 2024-04-24T22:40:12.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.382+0000 7f0875400640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f085006be30 0x7f085006e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0860005c00 tx=0x7f0860009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.382+0000 7f087abcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c3270 0x55ea803deb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f087abcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea80395e00 0x55ea80397d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f087abcc740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea80398250 0x55ea803c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f087abcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ea80299ed0 con 0x55ea803c3270 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f087abcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ea802fcdf0 con 0x55ea80395e00 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f087abcc740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55ea802a58d0 con 0x55ea80398250 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0875400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c3270 0x55ea803deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0875400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c3270 0x55ea803deb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57902/0 (socket says 172.21.15.47:57902) 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0875400640 1 -- 172.21.15.47:0/1111063853 learned_addr learned my addr 172.21.15.47:0/1111063853 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0874a00640 1 --2- 172.21.15.47:0/1111063853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea80395e00 0x55ea80397d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0875e00640 1 --2- 172.21.15.47:0/1111063853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea80398250 0x55ea803c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0875400640 1 -- 172.21.15.47:0/1111063853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea80398250 msgr2=0x55ea803c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0875400640 1 --2- 172.21.15.47:0/1111063853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea80398250 0x55ea803c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0875400640 1 -- 172.21.15.47:0/1111063853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea80395e00 msgr2=0x55ea80397d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0875400640 1 --2- 172.21.15.47:0/1111063853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea80395e00 0x55ea80397d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0875400640 1 -- 172.21.15.47:0/1111063853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea800112d0 con 0x55ea803c3270 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.383+0000 7f0874a00640 1 --2- 172.21.15.47:0/1111063853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea80395e00 0x55ea80397d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.384+0000 7f0875400640 1 --2- 172.21.15.47:0/1111063853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c3270 0x55ea803deb10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0860015720 tx=0x7f0860001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad0e6a8c66b956c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.384+0000 7f0865c00640 1 -- 172.21.15.47:0/1111063853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860025070 con 0x55ea803c3270 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.384+0000 7f0865c00640 1 -- 172.21.15.47:0/1111063853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0860013700 con 0x55ea803c3270 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.384+0000 7f0865c00640 1 -- 172.21.15.47:0/1111063853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860013ad0 con 0x55ea803c3270 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.384+0000 7f087abcc740 1 -- 172.21.15.47:0/1111063853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c3270 msgr2=0x55ea803deb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.384+0000 7f087abcc740 1 --2- 172.21.15.47:0/1111063853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c3270 0x55ea803deb10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0860015720 tx=0x7f0860001250 comp rx=0 tx=0).stop 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.385+0000 7f087abcc740 1 -- 172.21.15.47:0/1111063853 shutdown_connections 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.385+0000 7f087abcc740 1 --2- 172.21.15.47:0/1111063853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea80398250 0x55ea803c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.385+0000 7f087abcc740 1 --2- 172.21.15.47:0/1111063853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea80395e00 0x55ea80397d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.385+0000 7f087abcc740 1 --2- 172.21.15.47:0/1111063853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c3270 0x55ea803deb10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.385+0000 7f087abcc740 1 -- 172.21.15.47:0/1111063853 >> 172.21.15.47:0/1111063853 conn(0x55ea80396c90 msgr2=0x55ea803de160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.385+0000 7f087abcc740 1 -- 172.21.15.47:0/1111063853 shutdown_connections 2024-04-24T22:40:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.385+0000 7f087abcc740 1 -- 172.21.15.47:0/1111063853 wait complete. 2024-04-24T22:40:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.385+0000 7f087abcc740 1 -- start start 2024-04-24T22:40:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.385+0000 7f087abcc740 1 Processor -- start 2024-04-24T22:40:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.386+0000 7f087abcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.386+0000 7f087abcc740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 0x55ea803c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.386+0000 7f087abcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea803c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.386+0000 7f087abcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ea802b8d20 con 0x55ea803c6960 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.386+0000 7f087abcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ea802f8db0 con 0x55ea803c3270 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.386+0000 7f087abcc740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55ea80396290 con 0x55ea803c83d0 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f087abcc740 10 client.0 _ll_register_callbacks cb 0x55ea802f3f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea803c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803ebe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:47674/0 (socket says 172.21.15.47:47674) 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875400640 1 -- 172.21.15.47:0/721732284 learned_addr learned my addr 172.21.15.47:0/721732284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0874a00640 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 0x55ea803c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875400640 1 -- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 msgr2=0x55ea803c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875400640 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 0x55ea803c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875400640 1 -- 172.21.15.47:0/721732284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 msgr2=0x55ea803c0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875400640 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea803c0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875400640 1 -- 172.21.15.47:0/721732284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea800db6b0 con 0x55ea803c3270 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875e00640 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea803c0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f0875400640 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803ebe60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f08600155c0 tx=0x7f0860000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52f988d82d297225 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.387+0000 7f085a000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f085a000640 1 -- 172.21.15.47:0/721732284 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860030020 con 0x55ea803c3270 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f087abcc740 1 -- 172.21.15.47:0/721732284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0860013700 con 0x55ea803c3270 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f085a000640 1 -- 172.21.15.47:0/721732284 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0860013d40 con 0x55ea803c3270 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f085a000640 1 -- 172.21.15.47:0/721732284 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f086003a050 con 0x55ea803c3270 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f087abcc740 10 client.4714 fetch_fsmap learned FSMap version 17 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f087abcc740 1 -- 172.21.15.47:0/721732284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55ea803c6440 con 0x55ea803c3270 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f085a000640 1 -- 172.21.15.47:0/721732284 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0860013700 con 0x55ea803c3270 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f085a000640 1 -- 172.21.15.47:0/721732284 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f0860036710 con 0x55ea803c3270 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f087abcc740 10 client.4714 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T22:40:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f087abcc740 10 client.4714 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f087abcc740 20 client.4714 populate_metadata read hostname 'smithi047' 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f087abcc740 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:6834/3103107210,v1:172.21.15.113:6837/3103107210] conn(0x55ea803eeb50 0x55ea803faa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.388+0000 7f087abcc740 4 client.4714 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.389+0000 7f087abcc740 1 -- 172.21.15.47:0/721732284 --> [v2:172.21.15.113:6834/3103107210,v1:172.21.15.113:6837/3103107210] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea8030a330 con 0x55ea803eeb50 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.389+0000 7f0874a00640 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:6834/3103107210,v1:172.21.15.113:6837/3103107210] conn(0x55ea803eeb50 0x55ea803faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.389+0000 7f0874a00640 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:6834/3103107210,v1:172.21.15.113:6837/3103107210] conn(0x55ea803eeb50 0x55ea803faa20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4132b6ec4c44edb1 server_cookie=600531d2ea00c915 in_seq=0 out_seq=0 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.389+0000 7f085a000640 10 client.4714 ms_handle_connect on v2:172.21.15.113:6834/3103107210 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f085a000640 1 -- 172.21.15.47:0/721732284 <== mds.2 v2:172.21.15.113:6834/3103107210 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea8030a330 con 0x55ea803eeb50 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f085a000640 10 client.4714 handle_command_reply: tid=0 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f087abcc740 1 client.4714 shutdown 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f087abcc740 1 -- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 msgr2=0x55ea803ebe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f087abcc740 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803ebe60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f08600155c0 tx=0x7f0860000f40 comp rx=0 tx=0).stop 2024-04-24T22:40:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f087abcc740 1 -- 172.21.15.47:0/721732284 shutdown_connections 2024-04-24T22:40:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f087abcc740 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:6834/3103107210,v1:172.21.15.113:6837/3103107210] conn(0x55ea803eeb50 0x55ea803faa20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f087abcc740 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea803c0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f087abcc740 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 0x55ea803c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f087abcc740 1 --2- 172.21.15.47:0/721732284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803ebe60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f085a000640 0 client.4714 ms_handle_reset on v2:172.21.15.113:6834/3103107210 2024-04-24T22:40:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.390+0000 7f087abcc740 1 -- 172.21.15.47:0/721732284 >> 172.21.15.47:0/721732284 conn(0x55ea80398250 msgr2=0x55ea8039a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.391+0000 7f087abcc740 1 -- 172.21.15.47:0/721732284 shutdown_connections 2024-04-24T22:40:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.391+0000 7f087abcc740 1 -- 172.21.15.47:0/721732284 wait complete. 2024-04-24T22:40:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.391+0000 7f087abcc740 20 client.4714 trim_cache size 0 max 16384 2024-04-24T22:40:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.410+0000 7f087abcc740 1 Processor -- start 2024-04-24T22:40:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.410+0000 7f087abcc740 1 -- start start 2024-04-24T22:40:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f087abcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f087abcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea80393890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f087abcc740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 0x55ea80393dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f087abcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f08701516b0 con 0x55ea803c6960 2024-04-24T22:40:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f087abcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ea803ddce0 con 0x55ea803c3270 2024-04-24T22:40:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f087abcc740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55ea803c6de0 con 0x55ea803c83d0 2024-04-24T22:40:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f0874a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea80393890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f0874a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea80393890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57906/0 (socket says 172.21.15.47:57906) 2024-04-24T22:40:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f0874a00640 1 -- 172.21.15.47:0/509845969 learned_addr learned my addr 172.21.15.47:0/509845969 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f0875400640 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.411+0000 7f0875e00640 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 0x55ea80393dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0874a00640 1 -- 172.21.15.47:0/509845969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 msgr2=0x55ea80393dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0874a00640 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 0x55ea80393dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0874a00640 1 -- 172.21.15.47:0/509845969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 msgr2=0x55ea803c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0874a00640 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0874a00640 1 -- 172.21.15.47:0/509845969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea803c7b30 con 0x55ea803c6960 2024-04-24T22:40:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0875400640 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0875e00640 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 0x55ea80393dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:40:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0874a00640 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea80393890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f085c055480 tx=0x7f085c009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be89adad87c9ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0865200640 1 -- 172.21.15.47:0/509845969 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c021040 con 0x55ea803c6960 2024-04-24T22:40:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0865200640 1 -- 172.21.15.47:0/509845969 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c0556e0 con 0x55ea803c6960 2024-04-24T22:40:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.412+0000 7f0865200640 1 -- 172.21.15.47:0/509845969 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c082da0 con 0x55ea803c6960 2024-04-24T22:40:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.413+0000 7f087abcc740 1 -- 172.21.15.47:0/509845969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 msgr2=0x55ea80393890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.413+0000 7f087abcc740 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea80393890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f085c055480 tx=0x7f085c009760 comp rx=0 tx=0).stop 2024-04-24T22:40:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.413+0000 7f087abcc740 1 -- 172.21.15.47:0/509845969 shutdown_connections 2024-04-24T22:40:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.413+0000 7f087abcc740 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c83d0 0x55ea80393dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.413+0000 7f087abcc740 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c6960 0x55ea80393890 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.413+0000 7f087abcc740 1 --2- 172.21.15.47:0/509845969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c3270 0x55ea803c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.413+0000 7f087abcc740 1 -- 172.21.15.47:0/509845969 >> 172.21.15.47:0/509845969 conn(0x55ea80398250 msgr2=0x55ea80398840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.413+0000 7f087abcc740 1 -- 172.21.15.47:0/509845969 shutdown_connections 2024-04-24T22:40:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.413+0000 7f087abcc740 1 -- 172.21.15.47:0/509845969 wait complete. 2024-04-24T22:40:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.413+0000 7f087abcc740 1 -- start start 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.414+0000 7f087abcc740 1 Processor -- start 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f087abcc740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c3270 0x55ea803e8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f087abcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c6960 0x55ea803ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f087abcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c83d0 0x55ea8044f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f087abcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ea803ef020 con 0x55ea803c83d0 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f087abcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ea803ee9d0 con 0x55ea803c6960 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f087abcc740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55ea803ef6d0 con 0x55ea803c3270 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f087abcc740 10 client.0 _ll_register_callbacks cb 0x7f0860023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0875e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c83d0 0x55ea8044f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0875e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c83d0 0x55ea8044f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57916/0 (socket says 172.21.15.47:57916) 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0874a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c6960 0x55ea803ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0875e00640 1 -- 172.21.15.47:0/354062679 learned_addr learned my addr 172.21.15.47:0/354062679 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0875400640 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c3270 0x55ea803e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0875400640 1 -- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c6960 msgr2=0x55ea803ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0875400640 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c6960 0x55ea803ff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0875400640 1 -- 172.21.15.47:0/354062679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c83d0 msgr2=0x55ea8044f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0875400640 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c83d0 0x55ea8044f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0875400640 1 -- 172.21.15.47:0/354062679 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55ea8039bbf0 con 0x55ea803c3270 2024-04-24T22:40:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.415+0000 7f0875e00640 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c83d0 0x55ea8044f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f0875400640 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c3270 0x55ea803e8260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f086000fb60 tx=0x7f086001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec3b25d90d7ac02d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f085a000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3301/0 2024-04-24T22:40:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f085a000640 1 -- 172.21.15.47:0/354062679 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860043060 con 0x55ea803c3270 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f085a000640 1 -- 172.21.15.47:0/354062679 <== mon.2 v2:172.21.15.113:3301/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0860004b60 con 0x55ea803c3270 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f087abcc740 1 -- 172.21.15.47:0/354062679 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f085c0556e0 con 0x55ea803c3270 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f085a000640 1 -- 172.21.15.47:0/354062679 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0860030040 con 0x55ea803c3270 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f087abcc740 10 client.4580 fetch_fsmap learned FSMap version 17 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f087abcc740 1 -- 172.21.15.47:0/354062679 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55ea80456600 con 0x55ea803c3270 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f085a000640 1 -- 172.21.15.47:0/354062679 <== mon.2 v2:172.21.15.113:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c0556e0 con 0x55ea803c3270 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f085a000640 1 -- 172.21.15.47:0/354062679 <== mon.2 v2:172.21.15.113:3301/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f0860013b40 con 0x55ea803c3270 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f087abcc740 10 client.4580 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f087abcc740 10 client.4580 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.416+0000 7f087abcc740 20 client.4580 populate_metadata read hostname 'smithi047' 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.417+0000 7f087abcc740 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:6834/3103107210,v1:172.21.15.113:6837/3103107210] conn(0x55ea80459570 0x55ea80461f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.417+0000 7f087abcc740 4 client.4580 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.417+0000 7f087abcc740 1 -- 172.21.15.47:0/354062679 --> [v2:172.21.15.113:6834/3103107210,v1:172.21.15.113:6837/3103107210] -- command(tid 0: {"prefix": "status"}) -- 0x55ea80304680 con 0x55ea80459570 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.417+0000 7f0874a00640 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:6834/3103107210,v1:172.21.15.113:6837/3103107210] conn(0x55ea80459570 0x55ea80461f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.417+0000 7f0874a00640 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:6834/3103107210,v1:172.21.15.113:6837/3103107210] conn(0x55ea80459570 0x55ea80461f60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=637cf120b1830330 server_cookie=69b308012ada87f4 in_seq=0 out_seq=0 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.417+0000 7f085a000640 10 client.4580 ms_handle_connect on v2:172.21.15.113:6834/3103107210 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.417+0000 7f085a000640 1 -- 172.21.15.47:0/354062679 <== mds.2 v2:172.21.15.113:6834/3103107210 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ea80304680 con 0x55ea80459570 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.417+0000 7f085a000640 10 client.4580 handle_command_reply: tid=0 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.417+0000 7f087abcc740 1 client.4580 shutdown 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 1 -- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c3270 msgr2=0x55ea803e8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c3270 0x55ea803e8260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f086000fb60 tx=0x7f086001d000 comp rx=0 tx=0).stop 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 1 -- 172.21.15.47:0/354062679 shutdown_connections 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:6834/3103107210,v1:172.21.15.113:6837/3103107210] conn(0x55ea80459570 0x55ea80461f60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ea803c83d0 0x55ea8044f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea803c6960 0x55ea803ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 1 --2- 172.21.15.47:0/354062679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ea803c3270 0x55ea803e8260 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f085a000640 0 client.4580 ms_handle_reset on v2:172.21.15.113:6834/3103107210 2024-04-24T22:40:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 1 -- 172.21.15.47:0/354062679 >> 172.21.15.47:0/354062679 conn(0x55ea80398250 msgr2=0x55ea803c4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 1 -- 172.21.15.47:0/354062679 shutdown_connections 2024-04-24T22:40:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 1 -- 172.21.15.47:0/354062679 wait complete. 2024-04-24T22:40:12.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f087abcc740 20 client.4580 trim_cache size 0 max 16384 2024-04-24T22:40:12.428 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:40:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 2, 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4279, 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 23.650725609999999, 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 17, 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 21, 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 21, 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 42.791755250000001 2024-04-24T22:40:12.429 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:40:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f0876800640 1 -- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f085006be30 msgr2=0x7f085006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f0876800640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f085006be30 0x7f085006e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0860005c00 tx=0x7f0860009040 comp rx=0 tx=0).stop 2024-04-24T22:40:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f0876800640 1 -- 172.21.15.47:0/784121121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0870152670 msgr2=0x7f0870159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.418+0000 7f0876800640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0870152670 0x7f0870159d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f085c00ebe0 tx=0x7f085c00c520 comp rx=0 tx=0).stop 2024-04-24T22:40:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.419+0000 7f0876800640 1 -- 172.21.15.47:0/784121121 shutdown_connections 2024-04-24T22:40:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.419+0000 7f0876800640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f085006be30 0x7f085006e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.419+0000 7f0876800640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0870158760 0x7f0870150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.419+0000 7f0876800640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0870152670 0x7f0870159d50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.419+0000 7f0876800640 1 --2- 172.21.15.47:0/784121121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f087006b3d0 0x7f0870144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.419+0000 7f0876800640 1 -- 172.21.15.47:0/784121121 >> 172.21.15.47:0/784121121 conn(0x7f08700c9410 msgr2=0x7f0870157520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.419+0000 7f0876800640 1 -- 172.21.15.47:0/784121121 shutdown_connections 2024-04-24T22:40:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.419+0000 7f0876800640 1 -- 172.21.15.47:0/784121121 wait complete. 2024-04-24T22:40:12.439 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T22:40:12.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.640+0000 7fa137e00640 1 Processor -- start 2024-04-24T22:40:12.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.641+0000 7fa137e00640 1 -- start start 2024-04-24T22:40:12.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.642+0000 7fa137e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa130151780 0x7fa1300c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa137e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa13015aa30 con 0x7fa130073670 2024-04-24T22:40:12.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa137e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa13015ad10 con 0x7fa1300c1a00 2024-04-24T22:40:12.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa137e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa13015aff0 con 0x7fa130151780 2024-04-24T22:40:12.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa134a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa1300c1a00 0x7fa130072f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:54266/0 (socket says 172.21.15.47:54266) 2024-04-24T22:40:12.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa134a00640 1 -- 172.21.15.47:0/936444914 learned_addr learned my addr 172.21.15.47:0/936444914 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:12.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa135400640 1 --2- 172.21.15.47:0/936444914 >> v2:172.21.15.47:3300/0 conn(0x7fa130151780 0x7fa1300c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa12fe00640 1 -- 172.21.15.47:0/936444914 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031928031 0 0) 0x7fa13015ad10 con 0x7fa1300c1a00 2024-04-24T22:40:12.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa12fe00640 1 -- 172.21.15.47:0/936444914 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa114003520 con 0x7fa1300c1a00 2024-04-24T22:40:12.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa135400640 1 -- 172.21.15.47:0/936444914 >> v1:172.21.15.113:6789/0 conn(0x7fa1300c1a00 legacy=0x7fa130072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa135400640 1 -- 172.21.15.47:0/936444914 >> v1:172.21.15.47:6789/0 conn(0x7fa130073670 legacy=0x7fa1301576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.643+0000 7fa135400640 1 -- 172.21.15.47:0/936444914 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13015b2d0 con 0x7fa130151780 2024-04-24T22:40:12.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.644+0000 7fa135400640 1 --2- 172.21.15.47:0/936444914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130151780 0x7fa1300c8590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa1200097a0 tx=0x7fa12002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4b33724910e5af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.645+0000 7fa12fe00640 1 -- 172.21.15.47:0/936444914 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120037070 con 0x7fa130151780 2024-04-24T22:40:12.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.645+0000 7fa12fe00640 1 -- 172.21.15.47:0/936444914 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa120032640 con 0x7fa130151780 2024-04-24T22:40:12.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.645+0000 7fa12fe00640 1 -- 172.21.15.47:0/936444914 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120032a10 con 0x7fa130151780 2024-04-24T22:40:12.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.645+0000 7fa137e00640 1 -- 172.21.15.47:0/936444914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130151780 msgr2=0x7fa1300c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.645+0000 7fa137e00640 1 --2- 172.21.15.47:0/936444914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130151780 0x7fa1300c8590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa1200097a0 tx=0x7fa12002a2c0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.645+0000 7fa137e00640 1 -- 172.21.15.47:0/936444914 shutdown_connections 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.645+0000 7fa137e00640 1 --2- 172.21.15.47:0/936444914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130151780 0x7fa1300c8590 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.645+0000 7fa137e00640 1 -- 172.21.15.47:0/936444914 >> 172.21.15.47:0/936444914 conn(0x7fa13006e580 msgr2=0x7fa13006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.645+0000 7fa137e00640 1 -- 172.21.15.47:0/936444914 shutdown_connections 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.645+0000 7fa137e00640 1 -- 172.21.15.47:0/936444914 wait complete. 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.647+0000 7fa137e00640 1 Processor -- start 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.647+0000 7fa137e00640 1 -- start start 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa137e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130073670 0x7fa1301edf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa137e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1300c1a00 0x7fa130145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa137e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa130151780 0x7fa130147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa137e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa130157f80 con 0x7fa130073670 2024-04-24T22:40:12.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa137e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa130157e00 con 0x7fa130151780 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa137e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa1300c3410 con 0x7fa1300c1a00 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa135400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130073670 0x7fa1301edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa135400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130073670 0x7fa1301edf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57932/0 (socket says 172.21.15.47:57932) 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa135400640 1 -- 172.21.15.47:0/1490470087 learned_addr learned my addr 172.21.15.47:0/1490470087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa134a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1300c1a00 0x7fa130145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.648+0000 7fa135e00640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa130151780 0x7fa130147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.649+0000 7fa135400640 1 -- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1300c1a00 msgr2=0x7fa130145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.649+0000 7fa135400640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1300c1a00 0x7fa130145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.649+0000 7fa135400640 1 -- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa130151780 msgr2=0x7fa130147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.649+0000 7fa135400640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa130151780 0x7fa130147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.649+0000 7fa135400640 1 -- 172.21.15.47:0/1490470087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1300c3be0 con 0x7fa130073670 2024-04-24T22:40:12.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.649+0000 7fa135400640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130073670 0x7fa1301edf30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098c0 tx=0x7fa1200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.649+0000 7fa12d600640 1 -- 172.21.15.47:0/1490470087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120037040 con 0x7fa130073670 2024-04-24T22:40:12.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.650+0000 7fa12d600640 1 -- 172.21.15.47:0/1490470087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa120042070 con 0x7fa130073670 2024-04-24T22:40:12.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.650+0000 7fa12d600640 1 -- 172.21.15.47:0/1490470087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1200309b0 con 0x7fa130073670 2024-04-24T22:40:12.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.650+0000 7fa137e00640 1 -- 172.21.15.47:0/1490470087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1300c3640 con 0x7fa130073670 2024-04-24T22:40:12.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.650+0000 7fa137e00640 1 -- 172.21.15.47:0/1490470087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1301f15b0 con 0x7fa130073670 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.653+0000 7fa12d600640 1 -- 172.21.15.47:0/1490470087 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa120030b50 con 0x7fa130073670 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.653+0000 7fa12d600640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa11006be30 0x7fa11006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.654+0000 7fa12d600640 1 -- 172.21.15.47:0/1490470087 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa1200ab670 con 0x7fa130073670 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.654+0000 7fa134a00640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa11006be30 0x7fa11006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.654+0000 7fa13ab74740 1 Processor -- start 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.654+0000 7fa13ab74740 1 -- start start 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa134a00640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa11006be30 0x7fa11006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa130071030 tx=0x7fa11c007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d761a0 0x55fd61d732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d77690 0x55fd61d79e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 0x55fd61dc5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa13ab74740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fd61c77ed0 con 0x55fd61d77690 2024-04-24T22:40:12.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa13ab74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fd61cdadf0 con 0x55fd61d76580 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa13ab74740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fd61c838d0 con 0x55fd61d761a0 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa134a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d77690 0x55fd61d79e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa135400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d761a0 0x55fd61d732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa135e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 0x55fd61dc5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa134a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d77690 0x55fd61d79e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57948/0 (socket says 172.21.15.47:57948) 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.655+0000 7fa134a00640 1 -- 172.21.15.47:0/2612617534 learned_addr learned my addr 172.21.15.47:0/2612617534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.656+0000 7fa135e00640 1 -- 172.21.15.47:0/2612617534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d761a0 msgr2=0x55fd61d732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.656+0000 7fa135e00640 1 --2- 172.21.15.47:0/2612617534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d761a0 0x55fd61d732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.656+0000 7fa135e00640 1 -- 172.21.15.47:0/2612617534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d77690 msgr2=0x55fd61d79e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.656+0000 7fa135e00640 1 --2- 172.21.15.47:0/2612617534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d77690 0x55fd61d79e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.656+0000 7fa135e00640 1 -- 172.21.15.47:0/2612617534 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd619ef2d0 con 0x55fd61d76580 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.656+0000 7fa135400640 1 --2- 172.21.15.47:0/2612617534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d761a0 0x55fd61d732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.656+0000 7fa135e00640 1 --2- 172.21.15.47:0/2612617534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 0x55fd61dc5a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa1280012e0 tx=0x7fa1280015d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de68aaf3d9d5c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.656+0000 7fa125c00640 1 -- 172.21.15.47:0/2612617534 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128003b80 con 0x55fd61d76580 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.656+0000 7fa125c00640 1 -- 172.21.15.47:0/2612617534 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa128018070 con 0x55fd61d76580 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.656+0000 7fa125c00640 1 -- 172.21.15.47:0/2612617534 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12800f900 con 0x55fd61d76580 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.657+0000 7fa13ab74740 1 -- 172.21.15.47:0/2612617534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 msgr2=0x55fd61dc5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.657+0000 7fa13ab74740 1 --2- 172.21.15.47:0/2612617534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 0x55fd61dc5a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa1280012e0 tx=0x7fa1280015d0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.657+0000 7fa13ab74740 1 -- 172.21.15.47:0/2612617534 shutdown_connections 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.657+0000 7fa13ab74740 1 --2- 172.21.15.47:0/2612617534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 0x55fd61dc5a50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.657+0000 7fa13ab74740 1 --2- 172.21.15.47:0/2612617534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d77690 0x55fd61d79e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.657+0000 7fa13ab74740 1 --2- 172.21.15.47:0/2612617534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d761a0 0x55fd61d732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.657+0000 7fa13ab74740 1 -- 172.21.15.47:0/2612617534 >> 172.21.15.47:0/2612617534 conn(0x55fd61d74a80 msgr2=0x55fd61dbbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.657+0000 7fa13ab74740 1 -- 172.21.15.47:0/2612617534 shutdown_connections 2024-04-24T22:40:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.657+0000 7fa13ab74740 1 -- 172.21.15.47:0/2612617534 wait complete. 2024-04-24T22:40:12.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.658+0000 7fa13ab74740 1 -- start start 2024-04-24T22:40:12.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.658+0000 7fa13ab74740 1 Processor -- start 2024-04-24T22:40:12.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d76580 0x55fd61dbb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dbd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61db2730 0x55fd61da0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa13ab74740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fd61c96d20 con 0x55fd61d76580 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa13ab74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fd61cd6db0 con 0x55fd61d77690 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa13ab74740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fd61d742a0 con 0x55fd61db2730 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa13ab74740 10 client.0 _ll_register_callbacks cb 0x55fd61cd1f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa135400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d76580 0x55fd61dbb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa134a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dbd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa134a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dbd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:47742/0 (socket says 172.21.15.47:47742) 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa134a00640 1 -- 172.21.15.47:0/870745316 learned_addr learned my addr 172.21.15.47:0/870745316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.659+0000 7fa135e00640 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61db2730 0x55fd61da0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.660+0000 7fa134a00640 1 -- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61db2730 msgr2=0x55fd61da0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.660+0000 7fa134a00640 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61db2730 0x55fd61da0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.660+0000 7fa134a00640 1 -- 172.21.15.47:0/870745316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d76580 msgr2=0x55fd61dbb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.660+0000 7fa134a00640 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d76580 0x55fd61dbb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.660+0000 7fa134a00640 1 -- 172.21.15.47:0/870745316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd61ab96b0 con 0x55fd61d77690 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.660+0000 7fa134a00640 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dbd1a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0011e0 tx=0x7fa11c001150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d40f1fd909aea9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.660+0000 7fa11a000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.660+0000 7fa11a000640 1 -- 172.21.15.47:0/870745316 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c025020 con 0x55fd61d77690 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.660+0000 7fa11a000640 1 -- 172.21.15.47:0/870745316 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa11c013680 con 0x55fd61d77690 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.660+0000 7fa13ab74740 1 -- 172.21.15.47:0/870745316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa128018070 con 0x55fd61d77690 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa11a000640 1 -- 172.21.15.47:0/870745316 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa11c02f050 con 0x55fd61d77690 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa13ab74740 10 client.4732 fetch_fsmap learned FSMap version 17 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa13ab74740 1 -- 172.21.15.47:0/870745316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55fd61db0920 con 0x55fd61d77690 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa11a000640 1 -- 172.21.15.47:0/870745316 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa128018070 con 0x55fd61d77690 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa11a000640 1 -- 172.21.15.47:0/870745316 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fa11c032020 con 0x55fd61d77690 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa13ab74740 10 client.4732 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa13ab74740 10 client.4732 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T22:40:12.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa13ab74740 20 client.4732 populate_metadata read hostname 'smithi047' 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa13ab74740 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:6835/3621458364,v1:172.21.15.113:6838/3621458364] conn(0x55fd61dcd330 0x55fd61ddc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa13ab74740 4 client.4732 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa13ab74740 1 -- 172.21.15.47:0/870745316 --> [v2:172.21.15.113:6835/3621458364,v1:172.21.15.113:6838/3621458364] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd61ce8330 con 0x55fd61dcd330 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.661+0000 7fa135400640 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:6835/3621458364,v1:172.21.15.113:6838/3621458364] conn(0x55fd61dcd330 0x55fd61ddc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.662+0000 7fa135400640 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:6835/3621458364,v1:172.21.15.113:6838/3621458364] conn(0x55fd61dcd330 0x55fd61ddc0d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4765a5dfa77a398 server_cookie=baa3f820db61692d in_seq=0 out_seq=0 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.662+0000 7fa11a000640 10 client.4732 ms_handle_connect on v2:172.21.15.113:6835/3621458364 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa11a000640 1 -- 172.21.15.47:0/870745316 <== mds.1 v2:172.21.15.113:6835/3621458364 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd61ce8330 con 0x55fd61dcd330 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa11a000640 10 client.4732 handle_command_reply: tid=0 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 client.4732 shutdown 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 -- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 msgr2=0x55fd61dbd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dbd1a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0011e0 tx=0x7fa11c001150 comp rx=0 tx=0).stop 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 -- 172.21.15.47:0/870745316 shutdown_connections 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:6835/3621458364,v1:172.21.15.113:6838/3621458364] conn(0x55fd61dcd330 0x55fd61ddc0d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61db2730 0x55fd61da0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa11a000640 0 client.4732 ms_handle_reset on v2:172.21.15.113:6835/3621458364 2024-04-24T22:40:12.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dbd1a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 --2- 172.21.15.47:0/870745316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61d76580 0x55fd61dbb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 -- 172.21.15.47:0/870745316 >> 172.21.15.47:0/870745316 conn(0x55fd61d761a0 msgr2=0x55fd61d73900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 -- 172.21.15.47:0/870745316 shutdown_connections 2024-04-24T22:40:12.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 1 -- 172.21.15.47:0/870745316 wait complete. 2024-04-24T22:40:12.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.663+0000 7fa13ab74740 20 client.4732 trim_cache size 0 max 16384 2024-04-24T22:40:12.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.683+0000 7fa13ab74740 1 Processor -- start 2024-04-24T22:40:12.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.683+0000 7fa13ab74740 1 -- start start 2024-04-24T22:40:12.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 0x55fd61dd0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d77690 0x55fd61dd0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 0x55fd61da1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa13ab74740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa13006ba70 con 0x55fd61db2730 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa13ab74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fd61db2280 con 0x55fd61d76580 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa13ab74740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fd61dbd8f0 con 0x55fd61d77690 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa135400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 0x55fd61dd0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa134a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d77690 0x55fd61dd0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa135400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 0x55fd61dd0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:47758/0 (socket says 172.21.15.47:47758) 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa135400640 1 -- 172.21.15.47:0/1187895233 learned_addr learned my addr 172.21.15.47:0/1187895233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.684+0000 7fa134a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d77690 0x55fd61dd0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:44162/0 (socket says 172.21.15.47:44162) 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.685+0000 7fa135e00640 1 --2- 172.21.15.47:0/1187895233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 0x55fd61da1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.685+0000 7fa134a00640 1 -- 172.21.15.47:0/1187895233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 msgr2=0x55fd61dd0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.685+0000 7fa134a00640 1 --2- 172.21.15.47:0/1187895233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 0x55fd61dd0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.685+0000 7fa134a00640 1 -- 172.21.15.47:0/1187895233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 msgr2=0x55fd61da1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.685+0000 7fa134a00640 1 --2- 172.21.15.47:0/1187895233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 0x55fd61da1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.685+0000 7fa134a00640 1 -- 172.21.15.47:0/1187895233 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd61db2da0 con 0x55fd61d77690 2024-04-24T22:40:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.685+0000 7fa134a00640 1 --2- 172.21.15.47:0/1187895233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d77690 0x55fd61dd0e80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000f60 tx=0x7fa11c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2969b93512575cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.685+0000 7fa125200640 1 -- 172.21.15.47:0/1187895233 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c037070 con 0x55fd61d77690 2024-04-24T22:40:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.685+0000 7fa125200640 1 -- 172.21.15.47:0/1187895233 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11c01b070 con 0x55fd61d77690 2024-04-24T22:40:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.685+0000 7fa125200640 1 -- 172.21.15.47:0/1187895233 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c025070 con 0x55fd61d77690 2024-04-24T22:40:12.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.686+0000 7fa13ab74740 1 -- 172.21.15.47:0/1187895233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d77690 msgr2=0x55fd61dd0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.686+0000 7fa13ab74740 1 --2- 172.21.15.47:0/1187895233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d77690 0x55fd61dd0e80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000f60 tx=0x7fa11c01b000 comp rx=0 tx=0).stop 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.686+0000 7fa13ab74740 1 -- 172.21.15.47:0/1187895233 shutdown_connections 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.686+0000 7fa13ab74740 1 --2- 172.21.15.47:0/1187895233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 0x55fd61da1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.686+0000 7fa13ab74740 1 --2- 172.21.15.47:0/1187895233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d77690 0x55fd61dd0e80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.686+0000 7fa13ab74740 1 --2- 172.21.15.47:0/1187895233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d76580 0x55fd61dd0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.686+0000 7fa13ab74740 1 -- 172.21.15.47:0/1187895233 >> 172.21.15.47:0/1187895233 conn(0x55fd61d761a0 msgr2=0x55fd61dbc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.686+0000 7fa13ab74740 1 -- 172.21.15.47:0/1187895233 shutdown_connections 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.686+0000 7fa13ab74740 1 -- 172.21.15.47:0/1187895233 wait complete. 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.687+0000 7fa13ab74740 1 -- start start 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.687+0000 7fa13ab74740 1 Processor -- start 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d76580 0x55fd61dc6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa13ab74740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 0x55fd61dba600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa13ab74740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fd61dcd1b0 con 0x55fd61db2730 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa13ab74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fd61dd0490 con 0x55fd61d77690 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa13ab74740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fd61dcd880 con 0x55fd61d76580 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa13ab74740 10 client.0 _ll_register_callbacks cb 0x7fa11c006c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa135e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 0x55fd61dba600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa134a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa135400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d76580 0x55fd61dc6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa134a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dba0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:47772/0 (socket says 172.21.15.47:47772) 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.688+0000 7fa134a00640 1 -- 172.21.15.47:0/4194573571 learned_addr learned my addr 172.21.15.47:0/4194573571 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa134a00640 1 -- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d76580 msgr2=0x55fd61dc6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa134a00640 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d76580 0x55fd61dc6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa134a00640 1 -- 172.21.15.47:0/4194573571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 msgr2=0x55fd61dba600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa134a00640 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 0x55fd61dba600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa134a00640 1 -- 172.21.15.47:0/4194573571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd61db9e20 con 0x55fd61d77690 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa135400640 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d76580 0x55fd61dc6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa135e00640 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 0x55fd61dba600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa134a00640 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dba0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006cb0 tx=0x7fa11c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2b3bb9b8129416 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa11a000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa11a000640 1 -- 172.21.15.47:0/4194573571 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c025070 con 0x55fd61d77690 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa13ab74740 1 -- 172.21.15.47:0/4194573571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa11c01b070 con 0x55fd61d77690 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa11a000640 1 -- 172.21.15.47:0/4194573571 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa11c013680 con 0x55fd61d77690 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa11a000640 1 -- 172.21.15.47:0/4194573571 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa11c037040 con 0x55fd61d77690 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa13ab74740 10 client.4738 fetch_fsmap learned FSMap version 17 2024-04-24T22:40:12.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.689+0000 7fa13ab74740 1 -- 172.21.15.47:0/4194573571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55fd61e38f50 con 0x55fd61d77690 2024-04-24T22:40:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.690+0000 7fa11a000640 1 -- 172.21.15.47:0/4194573571 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11c01b070 con 0x55fd61d77690 2024-04-24T22:40:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.690+0000 7fa11a000640 1 -- 172.21.15.47:0/4194573571 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fa11c03a070 con 0x55fd61d77690 2024-04-24T22:40:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.690+0000 7fa13ab74740 10 client.4738 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T22:40:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.690+0000 7fa13ab74740 10 client.4738 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T22:40:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.690+0000 7fa13ab74740 20 client.4738 populate_metadata read hostname 'smithi047' 2024-04-24T22:40:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.690+0000 7fa13ab74740 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:6835/3621458364,v1:172.21.15.113:6838/3621458364] conn(0x55fd61e386c0 0x55fd61e3bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.690+0000 7fa13ab74740 4 client.4738 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:40:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.690+0000 7fa13ab74740 1 -- 172.21.15.47:0/4194573571 --> [v2:172.21.15.113:6835/3621458364,v1:172.21.15.113:6838/3621458364] -- command(tid 0: {"prefix": "status"}) -- 0x55fd61ce2680 con 0x55fd61e386c0 2024-04-24T22:40:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.690+0000 7fa135400640 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:6835/3621458364,v1:172.21.15.113:6838/3621458364] conn(0x55fd61e386c0 0x55fd61e3bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.691+0000 7fa135400640 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:6835/3621458364,v1:172.21.15.113:6838/3621458364] conn(0x55fd61e386c0 0x55fd61e3bb00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3de00c19892021f server_cookie=ef2fffe833d7788b in_seq=0 out_seq=0 2024-04-24T22:40:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.691+0000 7fa11a000640 10 client.4738 ms_handle_connect on v2:172.21.15.113:6835/3621458364 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.691+0000 7fa11a000640 1 -- 172.21.15.47:0/4194573571 <== mds.1 v2:172.21.15.113:6835/3621458364 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55fd61ce2680 con 0x55fd61e386c0 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.691+0000 7fa11a000640 10 client.4738 handle_command_reply: tid=0 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.691+0000 7fa13ab74740 1 client.4738 shutdown 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 1 -- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 msgr2=0x55fd61dba0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dba0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006cb0 tx=0x7fa11c015000 comp rx=0 tx=0).stop 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 1 -- 172.21.15.47:0/4194573571 shutdown_connections 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:6835/3621458364,v1:172.21.15.113:6838/3621458364] conn(0x55fd61e386c0 0x55fd61e3bb00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fd61db2730 0x55fd61dba600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd61d77690 0x55fd61dba0c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 1 --2- 172.21.15.47:0/4194573571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd61d76580 0x55fd61dc6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 1 -- 172.21.15.47:0/4194573571 >> 172.21.15.47:0/4194573571 conn(0x55fd61d761a0 msgr2=0x55fd61dd08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa11a000640 0 client.4738 ms_handle_reset on v2:172.21.15.113:6835/3621458364 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 1 -- 172.21.15.47:0/4194573571 shutdown_connections 2024-04-24T22:40:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 1 -- 172.21.15.47:0/4194573571 wait complete. 2024-04-24T22:40:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.692+0000 7fa13ab74740 20 client.4738 trim_cache size 0 max 16384 2024-04-24T22:40:12.699 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:40:12.699 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:40:12.699 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 1, 2024-04-24T22:40:12.699 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4282, 2024-04-24T22:40:12.699 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-24T22:40:12.699 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-24T22:40:12.699 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-24T22:40:12.699 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 28.951706322, 2024-04-24T22:40:12.699 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 17, 2024-04-24T22:40:12.700 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 21, 2024-04-24T22:40:12.700 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 21, 2024-04-24T22:40:12.700 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 43.065115835999997 2024-04-24T22:40:12.700 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:40:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 -- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa11006be30 msgr2=0x7fa11006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa11006be30 0x7fa11006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa130071030 tx=0x7fa11c007d30 comp rx=0 tx=0).stop 2024-04-24T22:40:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 -- 172.21.15.47:0/1490470087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130073670 msgr2=0x7fa1301edf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130073670 0x7fa1301edf30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098c0 tx=0x7fa1200046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 -- 172.21.15.47:0/1490470087 shutdown_connections 2024-04-24T22:40:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa11006be30 0x7fa11006e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa130151780 0x7fa130147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1300c1a00 0x7fa130145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 --2- 172.21.15.47:0/1490470087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa130073670 0x7fa1301edf30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 -- 172.21.15.47:0/1490470087 >> 172.21.15.47:0/1490470087 conn(0x7fa13006e580 msgr2=0x7fa1300c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 -- 172.21.15.47:0/1490470087 shutdown_connections 2024-04-24T22:40:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.693+0000 7fa137e00640 1 -- 172.21.15.47:0/1490470087 wait complete. 2024-04-24T22:40:12.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T22:40:12.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.914+0000 7f29af200640 1 Processor -- start 2024-04-24T22:40:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.914+0000 7f29af200640 1 -- start start 2024-04-24T22:40:13.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.915+0000 7f29af200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f29b00c2850 0x7f29b006e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.915+0000 7f29af200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00c6880 0x7f29b0169160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29af200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b0162d40 con 0x7f29b006e1b0 2024-04-24T22:40:13.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29af200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f29b0161e70 con 0x7f29b00c2850 2024-04-24T22:40:13.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29af200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f29b00c71f0 con 0x7f29b00c6880 2024-04-24T22:40:13.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29ad400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f29b00c2850 0x7f29b006e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29ad400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f29b00c2850 0x7f29b006e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57982/0 (socket says 172.21.15.47:57982) 2024-04-24T22:40:13.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29ad400640 1 -- 172.21.15.47:0/599099954 learned_addr learned my addr 172.21.15.47:0/599099954 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:13.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29ae800640 1 --2- 172.21.15.47:0/599099954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00c6880 0x7f29b0169160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29aca00640 1 -- 172.21.15.47:0/599099954 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3479289337 0 0) 0x7f29a0001110 con 0x7f29b006e1b0 2024-04-24T22:40:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29aca00640 1 -- 172.21.15.47:0/599099954 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2990003520 con 0x7f29b006e1b0 2024-04-24T22:40:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29ad400640 1 -- 172.21.15.47:0/599099954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00c6880 msgr2=0x7f29b0169160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29ad400640 1 --2- 172.21.15.47:0/599099954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00c6880 0x7f29b0169160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29ad400640 1 -- 172.21.15.47:0/599099954 >> v1:172.21.15.113:6789/0 conn(0x7f29b006e1b0 legacy=0x7f29b0162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.916+0000 7f29ad400640 1 -- 172.21.15.47:0/599099954 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b00c7810 con 0x7f29b00c2850 2024-04-24T22:40:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.917+0000 7f29ad400640 1 --2- 172.21.15.47:0/599099954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c2850 0x7f29b006e590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2998009e50 tx=0x7f299802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13fe36c0ddb57ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.918+0000 7f29aca00640 1 -- 172.21.15.47:0/599099954 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998036070 con 0x7f29b00c2850 2024-04-24T22:40:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.918+0000 7f29aca00640 1 -- 172.21.15.47:0/599099954 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2998031600 con 0x7f29b00c2850 2024-04-24T22:40:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.918+0000 7f29aca00640 1 -- 172.21.15.47:0/599099954 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29980319d0 con 0x7f29b00c2850 2024-04-24T22:40:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.918+0000 7f29af200640 1 -- 172.21.15.47:0/599099954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c2850 msgr2=0x7f29b006e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.918+0000 7f29af200640 1 --2- 172.21.15.47:0/599099954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c2850 0x7f29b006e590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2998009e50 tx=0x7f299802a7a0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.919+0000 7f29af200640 1 -- 172.21.15.47:0/599099954 shutdown_connections 2024-04-24T22:40:13.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.919+0000 7f29af200640 1 --2- 172.21.15.47:0/599099954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00c6880 0x7f29b0169160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.919+0000 7f29af200640 1 --2- 172.21.15.47:0/599099954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c2850 0x7f29b006e590 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.919+0000 7f29af200640 1 -- 172.21.15.47:0/599099954 >> 172.21.15.47:0/599099954 conn(0x7f29b00cb900 msgr2=0x7f29b00cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:13.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.919+0000 7f29af200640 1 -- 172.21.15.47:0/599099954 shutdown_connections 2024-04-24T22:40:13.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.919+0000 7f29af200640 1 -- 172.21.15.47:0/599099954 wait complete. 2024-04-24T22:40:13.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.920+0000 7f29af200640 1 Processor -- start 2024-04-24T22:40:13.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.921+0000 7f29af200640 1 -- start start 2024-04-24T22:40:13.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.921+0000 7f29af200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29b006e1b0 0x7f29b015b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.921+0000 7f29af200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00c2850 0x7f29b01f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.921+0000 7f29af200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c6880 0x7f29b01f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.921+0000 7f29af200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f29b015fcd0 con 0x7f29b00c6880 2024-04-24T22:40:13.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.921+0000 7f29af200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00c7dd0 con 0x7f29b006e1b0 2024-04-24T22:40:13.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.921+0000 7f29af200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f29b0158e00 con 0x7f29b00c2850 2024-04-24T22:40:13.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.922+0000 7f29ae800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c6880 0x7f29b01f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.922+0000 7f29ae800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c6880 0x7f29b01f5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57998/0 (socket says 172.21.15.47:57998) 2024-04-24T22:40:13.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.922+0000 7f29ae800640 1 -- 172.21.15.47:0/29193472 learned_addr learned my addr 172.21.15.47:0/29193472 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:13.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.922+0000 7f29ad400640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00c2850 0x7f29b01f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.922+0000 7f29ade00640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29b006e1b0 0x7f29b015b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.922+0000 7f29ae800640 1 -- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00c2850 msgr2=0x7f29b01f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.922+0000 7f29ae800640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00c2850 0x7f29b01f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.922+0000 7f29ae800640 1 -- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29b006e1b0 msgr2=0x7f29b015b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.922+0000 7f29ae800640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29b006e1b0 0x7f29b015b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.922+0000 7f29ae800640 1 -- 172.21.15.47:0/29193472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b01f7eb0 con 0x7f29b00c6880 2024-04-24T22:40:13.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.923+0000 7f29ae800640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c6880 0x7f29b01f5430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e820 tx=0x7f29a800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:13.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.923+0000 7f29a6000640 1 -- 172.21.15.47:0/29193472 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8017070 con 0x7f29b00c6880 2024-04-24T22:40:13.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.923+0000 7f29a6000640 1 -- 172.21.15.47:0/29193472 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a800ee70 con 0x7f29b00c6880 2024-04-24T22:40:13.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.923+0000 7f29a6000640 1 -- 172.21.15.47:0/29193472 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a800cb60 con 0x7f29b00c6880 2024-04-24T22:40:13.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.923+0000 7f29af200640 1 -- 172.21.15.47:0/29193472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b01f8150 con 0x7f29b00c6880 2024-04-24T22:40:13.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.923+0000 7f29af200640 1 -- 172.21.15.47:0/29193472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b01f8710 con 0x7f29b00c6880 2024-04-24T22:40:13.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.926+0000 7f29a6000640 1 -- 172.21.15.47:0/29193472 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29a801a070 con 0x7f29b00c6880 2024-04-24T22:40:13.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.927+0000 7f29a6000640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f298c06be80 0x7f298c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.927+0000 7f29a6000640 1 -- 172.21.15.47:0/29193472 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f29a808acb0 con 0x7f29b00c6880 2024-04-24T22:40:13.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.927+0000 7f29ade00640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f298c06be80 0x7f298c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.928+0000 7f29b797e740 1 Processor -- start 2024-04-24T22:40:13.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.928+0000 7f29b797e740 1 -- start start 2024-04-24T22:40:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.928+0000 7f29ade00640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f298c06be80 0x7f298c06e340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f29a0007fd0 tx=0x7f29a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29b797e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d40b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29b797e740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7cf7e00 0x561ef7cf9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29b797e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7cfa250 0x561ef7d25650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29b797e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561ef7bfbed0 con 0x561ef7d25270 2024-04-24T22:40:13.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29b797e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561ef7c5edf0 con 0x561ef7cfa250 2024-04-24T22:40:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29b797e740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x561ef7c078d0 con 0x561ef7cf7e00 2024-04-24T22:40:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ade00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d40b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ae800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7cfa250 0x561ef7d25650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ad400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7cf7e00 0x561ef7cf9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ad400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7cf7e00 0x561ef7cf9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:44210/0 (socket says 172.21.15.47:44210) 2024-04-24T22:40:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ae800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7cfa250 0x561ef7d25650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:47786/0 (socket says 172.21.15.47:47786) 2024-04-24T22:40:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ad400640 1 -- 172.21.15.47:0/24656460 learned_addr learned my addr 172.21.15.47:0/24656460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ade00640 1 --2- 172.21.15.47:0/24656460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d40b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58014/0 (socket says 172.21.15.47:58014) 2024-04-24T22:40:13.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ae800640 1 -- 172.21.15.47:0/24656460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7cf7e00 msgr2=0x561ef7cf9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ae800640 1 --2- 172.21.15.47:0/24656460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7cf7e00 0x561ef7cf9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ae800640 1 -- 172.21.15.47:0/24656460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 msgr2=0x561ef7d40b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ae800640 1 --2- 172.21.15.47:0/24656460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d40b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ae800640 1 -- 172.21.15.47:0/24656460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef79732d0 con 0x561ef7cfa250 2024-04-24T22:40:13.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.929+0000 7f29ad400640 1 --2- 172.21.15.47:0/24656460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7cf7e00 0x561ef7cf9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.930+0000 7f29ae800640 1 --2- 172.21.15.47:0/24656460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7cfa250 0x561ef7d25650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f29a8058c60 tx=0x7f29a805d320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=787322f2bc2c370a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.930+0000 7f299e600640 1 -- 172.21.15.47:0/24656460 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8021040 con 0x561ef7cfa250 2024-04-24T22:40:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.930+0000 7f299e600640 1 -- 172.21.15.47:0/24656460 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a8057260 con 0x561ef7cfa250 2024-04-24T22:40:13.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.930+0000 7f299e600640 1 -- 172.21.15.47:0/24656460 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a800cd00 con 0x561ef7cfa250 2024-04-24T22:40:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.930+0000 7f29b797e740 1 -- 172.21.15.47:0/24656460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7cfa250 msgr2=0x561ef7d25650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.930+0000 7f29b797e740 1 --2- 172.21.15.47:0/24656460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7cfa250 0x561ef7d25650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f29a8058c60 tx=0x7f29a805d320 comp rx=0 tx=0).stop 2024-04-24T22:40:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.931+0000 7f29b797e740 1 -- 172.21.15.47:0/24656460 shutdown_connections 2024-04-24T22:40:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.931+0000 7f29b797e740 1 --2- 172.21.15.47:0/24656460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7cfa250 0x561ef7d25650 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.931+0000 7f29b797e740 1 --2- 172.21.15.47:0/24656460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7cf7e00 0x561ef7cf9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.931+0000 7f29b797e740 1 --2- 172.21.15.47:0/24656460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d40b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.931+0000 7f29b797e740 1 -- 172.21.15.47:0/24656460 >> 172.21.15.47:0/24656460 conn(0x561ef7cf8c90 msgr2=0x561ef7d40160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.931+0000 7f29b797e740 1 -- 172.21.15.47:0/24656460 shutdown_connections 2024-04-24T22:40:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.931+0000 7f29b797e740 1 -- 172.21.15.47:0/24656460 wait complete. 2024-04-24T22:40:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.931+0000 7f29b797e740 1 -- start start 2024-04-24T22:40:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.932+0000 7f29b797e740 1 Processor -- start 2024-04-24T22:40:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29b797e740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d25270 0x561ef7cf5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29b797e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 0x561ef7d22bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29b797e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d43910 0x561ef7d23130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29b797e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561ef7c1ad20 con 0x561ef7d412b0 2024-04-24T22:40:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29b797e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561ef7c5adb0 con 0x561ef7d43910 2024-04-24T22:40:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29b797e740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x561ef7cf8290 con 0x561ef7d25270 2024-04-24T22:40:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29b797e740 10 client.0 _ll_register_callbacks cb 0x561ef7c55f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:40:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29ad400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 0x561ef7d22bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29ad400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 0x561ef7d22bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58026/0 (socket says 172.21.15.47:58026) 2024-04-24T22:40:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29ad400640 1 -- 172.21.15.47:0/3917041182 learned_addr learned my addr 172.21.15.47:0/3917041182 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29ade00640 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d25270 0x561ef7cf5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29ae800640 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d43910 0x561ef7d23130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29ad400640 1 -- 172.21.15.47:0/3917041182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d25270 msgr2=0x561ef7cf5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29ad400640 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d25270 0x561ef7cf5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29ad400640 1 -- 172.21.15.47:0/3917041182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d43910 msgr2=0x561ef7d23130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29ad400640 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d43910 0x561ef7d23130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.933+0000 7f29ad400640 1 -- 172.21.15.47:0/3917041182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef7a3d6b0 con 0x561ef7d412b0 2024-04-24T22:40:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.934+0000 7f29ad400640 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 0x561ef7d22bf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2998009f70 tx=0x7f2998004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a182c21c9b7614 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.934+0000 7f2996a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-24T22:40:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.934+0000 7f2996a00640 1 -- 172.21.15.47:0/3917041182 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998042070 con 0x561ef7d412b0 2024-04-24T22:40:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.934+0000 7f2996a00640 1 -- 172.21.15.47:0/3917041182 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2998031bc0 con 0x561ef7d412b0 2024-04-24T22:40:13.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.934+0000 7f29b797e740 1 -- 172.21.15.47:0/3917041182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29a8057260 con 0x561ef7d412b0 2024-04-24T22:40:13.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.934+0000 7f2996a00640 1 -- 172.21.15.47:0/3917041182 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f299804e050 con 0x561ef7d412b0 2024-04-24T22:40:13.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.934+0000 7f29b797e740 10 client.4614 fetch_fsmap learned FSMap version 17 2024-04-24T22:40:13.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.934+0000 7f29b797e740 1 -- 172.21.15.47:0/3917041182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x561ef7d43570 con 0x561ef7d412b0 2024-04-24T22:40:13.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.935+0000 7f2996a00640 1 -- 172.21.15.47:0/3917041182 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a8057260 con 0x561ef7d412b0 2024-04-24T22:40:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.935+0000 7f2996a00640 1 -- 172.21.15.47:0/3917041182 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f299803e440 con 0x561ef7d412b0 2024-04-24T22:40:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.935+0000 7f29b797e740 10 client.4614 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T22:40:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.935+0000 7f29b797e740 10 client.4614 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T22:40:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.935+0000 7f29b797e740 20 client.4614 populate_metadata read hostname 'smithi047' 2024-04-24T22:40:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.935+0000 7f29b797e740 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.47:6834/1117134154,v1:172.21.15.47:6835/1117134154] conn(0x561ef7d506c0 0x561ef7d5c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.935+0000 7f29b797e740 4 client.4614 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T22:40:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.935+0000 7f29b797e740 1 -- 172.21.15.47:0/3917041182 --> [v2:172.21.15.47:6834/1117134154,v1:172.21.15.47:6835/1117134154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ef7c6c330 con 0x561ef7d506c0 2024-04-24T22:40:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.935+0000 7f29ade00640 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.47:6834/1117134154,v1:172.21.15.47:6835/1117134154] conn(0x561ef7d506c0 0x561ef7d5c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.936+0000 7f29ade00640 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.47:6834/1117134154,v1:172.21.15.47:6835/1117134154] conn(0x561ef7d506c0 0x561ef7d5c5c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f804c7ee285eb server_cookie=35bc45cee025d541 in_seq=0 out_seq=0 2024-04-24T22:40:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.936+0000 7f2996a00640 10 client.4614 ms_handle_connect on v2:172.21.15.47:6834/1117134154 2024-04-24T22:40:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f2996a00640 1 -- 172.21.15.47:0/3917041182 <== mds.0 v2:172.21.15.47:6834/1117134154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ef7c6c330 con 0x561ef7d506c0 2024-04-24T22:40:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f2996a00640 10 client.4614 handle_command_reply: tid=0 2024-04-24T22:40:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 client.4614 shutdown 2024-04-24T22:40:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 -- 172.21.15.47:0/3917041182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 msgr2=0x561ef7d22bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 0x561ef7d22bf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2998009f70 tx=0x7f2998004ab0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 -- 172.21.15.47:0/3917041182 shutdown_connections 2024-04-24T22:40:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.47:6834/1117134154,v1:172.21.15.47:6835/1117134154] conn(0x561ef7d506c0 0x561ef7d5c5c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d43910 0x561ef7d23130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 0x561ef7d22bf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 --2- 172.21.15.47:0/3917041182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d25270 0x561ef7cf5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f2996a00640 0 client.4614 ms_handle_reset on v2:172.21.15.47:6834/1117134154 2024-04-24T22:40:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 -- 172.21.15.47:0/3917041182 >> 172.21.15.47:0/3917041182 conn(0x561ef7cfa250 msgr2=0x561ef7cfc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 -- 172.21.15.47:0/3917041182 shutdown_connections 2024-04-24T22:40:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 1 -- 172.21.15.47:0/3917041182 wait complete. 2024-04-24T22:40:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.937+0000 7f29b797e740 20 client.4614 trim_cache size 0 max 16384 2024-04-24T22:40:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.958+0000 7f29b797e740 1 Processor -- start 2024-04-24T22:40:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.958+0000 7f29b797e740 1 -- start start 2024-04-24T22:40:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29b797e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d25270 0x561ef7d21ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29b797e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 0x561ef7cf6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29b797e740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d43910 0x561ef7cf6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29b797e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0162930 con 0x561ef7d412b0 2024-04-24T22:40:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29b797e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561ef7d236d0 con 0x561ef7d25270 2024-04-24T22:40:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29b797e740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x561ef7d4de20 con 0x561ef7d43910 2024-04-24T22:40:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29ad400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 0x561ef7cf6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29ade00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d25270 0x561ef7d21ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29ade00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d25270 0x561ef7d21ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:47802/0 (socket says 172.21.15.47:47802) 2024-04-24T22:40:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29ade00640 1 -- 172.21.15.47:0/2907483488 learned_addr learned my addr 172.21.15.47:0/2907483488 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29ae800640 1 --2- 172.21.15.47:0/2907483488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d43910 0x561ef7cf6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29ade00640 1 -- 172.21.15.47:0/2907483488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d43910 msgr2=0x561ef7cf6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29ade00640 1 --2- 172.21.15.47:0/2907483488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d43910 0x561ef7cf6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29ade00640 1 -- 172.21.15.47:0/2907483488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 msgr2=0x561ef7cf6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29ade00640 1 --2- 172.21.15.47:0/2907483488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 0x561ef7cf6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.959+0000 7f29ade00640 1 -- 172.21.15.47:0/2907483488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef7d24ad0 con 0x561ef7d25270 2024-04-24T22:40:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.960+0000 7f29ade00640 1 --2- 172.21.15.47:0/2907483488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d25270 0x561ef7d21ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f29a00012b0 tx=0x7f29a0012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a04a5dc2638860d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.960+0000 7f299dc00640 1 -- 172.21.15.47:0/2907483488 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0023910 con 0x561ef7d25270 2024-04-24T22:40:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.960+0000 7f299dc00640 1 -- 172.21.15.47:0/2907483488 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a001b040 con 0x561ef7d25270 2024-04-24T22:40:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.960+0000 7f299dc00640 1 -- 172.21.15.47:0/2907483488 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0023ab0 con 0x561ef7d25270 2024-04-24T22:40:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.961+0000 7f29b797e740 1 -- 172.21.15.47:0/2907483488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d25270 msgr2=0x561ef7d21ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.961+0000 7f29b797e740 1 --2- 172.21.15.47:0/2907483488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d25270 0x561ef7d21ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f29a00012b0 tx=0x7f29a0012000 comp rx=0 tx=0).stop 2024-04-24T22:40:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.961+0000 7f29b797e740 1 -- 172.21.15.47:0/2907483488 shutdown_connections 2024-04-24T22:40:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.961+0000 7f29b797e740 1 --2- 172.21.15.47:0/2907483488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d43910 0x561ef7cf6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.961+0000 7f29b797e740 1 --2- 172.21.15.47:0/2907483488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d412b0 0x561ef7cf6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.961+0000 7f29b797e740 1 --2- 172.21.15.47:0/2907483488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d25270 0x561ef7d21ef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.961+0000 7f29b797e740 1 -- 172.21.15.47:0/2907483488 >> 172.21.15.47:0/2907483488 conn(0x561ef7cfa250 msgr2=0x561ef7cf5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.961+0000 7f29b797e740 1 -- 172.21.15.47:0/2907483488 shutdown_connections 2024-04-24T22:40:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.961+0000 7f29b797e740 1 -- 172.21.15.47:0/2907483488 wait complete. 2024-04-24T22:40:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.961+0000 7f29b797e740 1 -- start start 2024-04-24T22:40:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.962+0000 7f29b797e740 1 Processor -- start 2024-04-24T22:40:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29b797e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29b797e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d412b0 0x561ef7cf6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29b797e740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d43910 0x561ef7db14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29b797e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561ef7d50b90 con 0x561ef7d25270 2024-04-24T22:40:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29b797e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561ef7d50540 con 0x561ef7d412b0 2024-04-24T22:40:13.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29b797e740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x561ef7d51240 con 0x561ef7d43910 2024-04-24T22:40:13.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29b797e740 10 client.0 _ll_register_callbacks cb 0x7f29a8009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T22:40:13.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29ade00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29ade00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58032/0 (socket says 172.21.15.47:58032) 2024-04-24T22:40:13.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29ade00640 1 -- 172.21.15.47:0/2715528588 learned_addr learned my addr 172.21.15.47:0/2715528588 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:13.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29ad400640 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d412b0 0x561ef7cf6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29ae800640 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d43910 0x561ef7db14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29ade00640 1 -- 172.21.15.47:0/2715528588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d43910 msgr2=0x561ef7db14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29ade00640 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d43910 0x561ef7db14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29ade00640 1 -- 172.21.15.47:0/2715528588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d412b0 msgr2=0x561ef7cf6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29ade00640 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d412b0 0x561ef7cf6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.963+0000 7f29ade00640 1 -- 172.21.15.47:0/2715528588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef7d3d040 con 0x561ef7d25270 2024-04-24T22:40:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.964+0000 7f29ade00640 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d605e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f29a0007fa0 tx=0x7f29a0001320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349e03ba1195fb09 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.964+0000 7f2996a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-24T22:40:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.964+0000 7f2996a00640 1 -- 172.21.15.47:0/2715528588 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0007040 con 0x561ef7d25270 2024-04-24T22:40:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.964+0000 7f2996a00640 1 -- 172.21.15.47:0/2715528588 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f29a00071e0 con 0x561ef7d25270 2024-04-24T22:40:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.964+0000 7f29b797e740 1 -- 172.21.15.47:0/2715528588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29a001b040 con 0x561ef7d25270 2024-04-24T22:40:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.964+0000 7f2996a00640 1 -- 172.21.15.47:0/2715528588 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f29a003c050 con 0x561ef7d25270 2024-04-24T22:40:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.964+0000 7f29b797e740 10 client.4620 fetch_fsmap learned FSMap version 17 2024-04-24T22:40:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.964+0000 7f29b797e740 1 -- 172.21.15.47:0/2715528588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x561ef7d57330 con 0x561ef7d25270 2024-04-24T22:40:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.964+0000 7f2996a00640 1 -- 172.21.15.47:0/2715528588 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a001b040 con 0x561ef7d25270 2024-04-24T22:40:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.965+0000 7f2996a00640 1 -- 172.21.15.47:0/2715528588 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f29a003f020 con 0x561ef7d25270 2024-04-24T22:40:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.965+0000 7f29b797e740 10 client.4620 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T22:40:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.965+0000 7f29b797e740 10 client.4620 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T22:40:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.965+0000 7f29b797e740 20 client.4620 populate_metadata read hostname 'smithi047' 2024-04-24T22:40:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.965+0000 7f29b797e740 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.47:6834/1117134154,v1:172.21.15.47:6835/1117134154] conn(0x561ef7d39b20 0x561ef7dc3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.965+0000 7f29b797e740 4 client.4620 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T22:40:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.965+0000 7f29b797e740 1 -- 172.21.15.47:0/2715528588 --> [v2:172.21.15.47:6834/1117134154,v1:172.21.15.47:6835/1117134154] -- command(tid 0: {"prefix": "status"}) -- 0x561ef7c66680 con 0x561ef7d39b20 2024-04-24T22:40:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.965+0000 7f29ad400640 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.47:6834/1117134154,v1:172.21.15.47:6835/1117134154] conn(0x561ef7d39b20 0x561ef7dc3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29ad400640 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.47:6834/1117134154,v1:172.21.15.47:6835/1117134154] conn(0x561ef7d39b20 0x561ef7dc3f90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c6ba2fb0b8cc54 server_cookie=9bc93d638298149a in_seq=0 out_seq=0 2024-04-24T22:40:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f2996a00640 10 client.4620 ms_handle_connect on v2:172.21.15.47:6834/1117134154 2024-04-24T22:40:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f2996a00640 1 -- 172.21.15.47:0/2715528588 <== mds.0 v2:172.21.15.47:6834/1117134154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x561ef7c66680 con 0x561ef7d39b20 2024-04-24T22:40:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f2996a00640 10 client.4620 handle_command_reply: tid=0 2024-04-24T22:40:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 client.4620 shutdown 2024-04-24T22:40:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 -- 172.21.15.47:0/2715528588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 msgr2=0x561ef7d605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d605e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f29a0007fa0 tx=0x7f29a0001320 comp rx=0 tx=0).stop 2024-04-24T22:40:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 -- 172.21.15.47:0/2715528588 shutdown_connections 2024-04-24T22:40:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.47:6834/1117134154,v1:172.21.15.47:6835/1117134154] conn(0x561ef7d39b20 0x561ef7dc3f90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561ef7d43910 0x561ef7db14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ef7d412b0 0x561ef7cf6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 --2- 172.21.15.47:0/2715528588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef7d25270 0x561ef7d605e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f2996a00640 0 client.4620 ms_handle_reset on v2:172.21.15.47:6834/1117134154 2024-04-24T22:40:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 -- 172.21.15.47:0/2715528588 >> 172.21.15.47:0/2715528588 conn(0x561ef7cfa250 msgr2=0x561ef7d22bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 -- 172.21.15.47:0/2715528588 shutdown_connections 2024-04-24T22:40:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 1 -- 172.21.15.47:0/2715528588 wait complete. 2024-04-24T22:40:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.966+0000 7f29b797e740 20 client.4620 trim_cache size 0 max 16384 2024-04-24T22:40:13.106 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:40:13.106 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:40:13.106 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-24T22:40:13.106 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4306, 2024-04-24T22:40:13.106 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-24T22:40:13.107 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-24T22:40:13.107 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-24T22:40:13.107 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 35.271522457000003, 2024-04-24T22:40:13.107 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 17, 2024-04-24T22:40:13.107 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 21, 2024-04-24T22:40:13.107 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 21, 2024-04-24T22:40:13.108 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 43.286553871999999 2024-04-24T22:40:13.108 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:40:13.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 -- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f298c06be80 msgr2=0x7f298c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f298c06be80 0x7f298c06e340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f29a0007fd0 tx=0x7f29a0009040 comp rx=0 tx=0).stop 2024-04-24T22:40:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 -- 172.21.15.47:0/29193472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c6880 msgr2=0x7f29b01f5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c6880 0x7f29b01f5430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e820 tx=0x7f29a800ec20 comp rx=0 tx=0).stop 2024-04-24T22:40:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 -- 172.21.15.47:0/29193472 shutdown_connections 2024-04-24T22:40:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f298c06be80 0x7f298c06e340 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29b00c6880 0x7f29b01f5430 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b00c2850 0x7f29b01f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 --2- 172.21.15.47:0/29193472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29b006e1b0 0x7f29b015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 -- 172.21.15.47:0/29193472 >> 172.21.15.47:0/29193472 conn(0x7f29b00cb900 msgr2=0x7f29b00cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 -- 172.21.15.47:0/29193472 shutdown_connections 2024-04-24T22:40:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:12.967+0000 7f29af200640 1 -- 172.21.15.47:0/29193472 wait complete. 2024-04-24T22:40:13.112 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T22:40:13.125 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T22:40:13.125 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T22:40:13.125 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T22:40:13.126 INFO:teuthology.run_tasks:Running task kclient... 2024-04-24T22:40:13.154 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T22:40:13.154 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-24T22:40:13.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:40:13.155 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-24T22:40:13.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:40:13.220 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-24T22:40:13.366 INFO:teuthology.orchestra.run.smithi047.stderr:Cannot find device "ceph-brx" 2024-04-24T22:40:13.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:40:13.371 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-24T22:40:13.371 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T22:40:13.371 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T22:40:13.372 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T22:40:13.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.587+0000 7feb0ac00640 1 Processor -- start 2024-04-24T22:40:13.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.587+0000 7feb0ac00640 1 -- start start 2024-04-24T22:40:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.589+0000 7feb0ac00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb0415ad80 0x7feb0406b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.590+0000 7feb0ac00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0406c360 con 0x7feb040c81d0 2024-04-24T22:40:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.590+0000 7feb0ac00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0406c640 con 0x7feb041549c0 2024-04-24T22:40:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.590+0000 7feb0ac00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7feb040730e0 con 0x7feb0415ad80 2024-04-24T22:40:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.590+0000 7feafbe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feb040c81d0 0x7feb0415a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51978/0 (socket says 172.21.15.47:51978) 2024-04-24T22:40:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.590+0000 7feafbe00640 1 -- 172.21.15.47:0/577729267 learned_addr learned my addr 172.21.15.47:0/577729267 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.590+0000 7feb08c00640 1 --2- 172.21.15.47:0/577729267 >> v2:172.21.15.113:3300/0 conn(0x7feb0415ad80 0x7feb0406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.590+0000 7feb03400640 1 -- 172.21.15.47:0/577729267 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1139241121 0 0) 0x7feb0406c360 con 0x7feb040c81d0 2024-04-24T22:40:13.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.590+0000 7feb03400640 1 -- 172.21.15.47:0/577729267 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feae8003520 con 0x7feb040c81d0 2024-04-24T22:40:13.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.590+0000 7feb03400640 1 -- 172.21.15.47:0/577729267 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3299566966 0 0) 0x7feb0406c640 con 0x7feb041549c0 2024-04-24T22:40:13.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.590+0000 7feb03400640 1 -- 172.21.15.47:0/577729267 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb0406c360 con 0x7feb041549c0 2024-04-24T22:40:13.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.591+0000 7feb08c00640 1 -- 172.21.15.47:0/577729267 >> v1:172.21.15.113:6789/0 conn(0x7feb041549c0 legacy=0x7feb04066ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.591+0000 7feb08c00640 1 -- 172.21.15.47:0/577729267 >> v1:172.21.15.47:6789/0 conn(0x7feb040c81d0 legacy=0x7feb0415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.591+0000 7feb08c00640 1 -- 172.21.15.47:0/577729267 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb04073330 con 0x7feb0415ad80 2024-04-24T22:40:13.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.592+0000 7feb08c00640 1 --2- 172.21.15.47:0/577729267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb0415ad80 0x7feb0406b440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feafc009d90 tx=0x7feafc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68371821d147d153 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:13.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.592+0000 7feb03400640 1 -- 172.21.15.47:0/577729267 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc007740 con 0x7feb0415ad80 2024-04-24T22:40:13.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.592+0000 7feb03400640 1 -- 172.21.15.47:0/577729267 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feafc0078e0 con 0x7feb0415ad80 2024-04-24T22:40:13.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.592+0000 7feb03400640 1 -- 172.21.15.47:0/577729267 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc007cb0 con 0x7feb0415ad80 2024-04-24T22:40:13.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.592+0000 7feb0ac00640 1 -- 172.21.15.47:0/577729267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb0415ad80 msgr2=0x7feb0406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.592+0000 7feb0ac00640 1 --2- 172.21.15.47:0/577729267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb0415ad80 0x7feb0406b440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feafc009d90 tx=0x7feafc02a6f0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.593+0000 7feb0ac00640 1 -- 172.21.15.47:0/577729267 shutdown_connections 2024-04-24T22:40:13.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.593+0000 7feb0ac00640 1 --2- 172.21.15.47:0/577729267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb0415ad80 0x7feb0406b440 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.593+0000 7feb0ac00640 1 -- 172.21.15.47:0/577729267 >> 172.21.15.47:0/577729267 conn(0x7feb040c9cb0 msgr2=0x7feb040ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:13.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.593+0000 7feb0ac00640 1 -- 172.21.15.47:0/577729267 shutdown_connections 2024-04-24T22:40:13.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.593+0000 7feb0ac00640 1 -- 172.21.15.47:0/577729267 wait complete. 2024-04-24T22:40:13.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.595+0000 7feb0ac00640 1 Processor -- start 2024-04-24T22:40:13.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.595+0000 7feb0ac00640 1 -- start start 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb0ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb040c81d0 0x7feb041f0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb0ac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb041549c0 0x7feb04159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb0ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0415ad80 0x7feb0415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb0ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb040675b0 con 0x7feb0415ad80 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb0ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feb040736a0 con 0x7feb040c81d0 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb0ac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7feb04067730 con 0x7feb041549c0 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb08c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0415ad80 0x7feb0415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb03e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb040c81d0 0x7feb041f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb08c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0415ad80 0x7feb0415d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58042/0 (socket says 172.21.15.47:58042) 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb08c00640 1 -- 172.21.15.47:0/2528511795 learned_addr learned my addr 172.21.15.47:0/2528511795 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:13.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feafbe00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb041549c0 0x7feb04159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.596+0000 7feb03e00640 1 -- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb041549c0 msgr2=0x7feb04159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb03e00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb041549c0 0x7feb04159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb03e00640 1 -- 172.21.15.47:0/2528511795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0415ad80 msgr2=0x7feb0415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb03e00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0415ad80 0x7feb0415d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb03e00640 1 -- 172.21.15.47:0/2528511795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0415d360 con 0x7feb040c81d0 2024-04-24T22:40:13.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb08c00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0415ad80 0x7feb0415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb03e00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb040c81d0 0x7feb041f0190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c3f0 tx=0x7feaf400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb00c00640 1 -- 172.21.15.47:0/2528511795 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4017070 con 0x7feb040c81d0 2024-04-24T22:40:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb00c00640 1 -- 172.21.15.47:0/2528511795 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf40052f0 con 0x7feb040c81d0 2024-04-24T22:40:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb00c00640 1 -- 172.21.15.47:0/2528511795 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4003830 con 0x7feb040c81d0 2024-04-24T22:40:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb0ac00640 1 -- 172.21.15.47:0/2528511795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb04150b30 con 0x7feb040c81d0 2024-04-24T22:40:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.597+0000 7feb0ac00640 1 -- 172.21.15.47:0/2528511795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0415b910 con 0x7feb040c81d0 2024-04-24T22:40:13.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.598+0000 7feb0ac00640 1 -- 172.21.15.47:0/2528511795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb04073b90 con 0x7feb040c81d0 2024-04-24T22:40:13.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.599+0000 7feb00c00640 1 -- 172.21.15.47:0/2528511795 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feaf401a070 con 0x7feb040c81d0 2024-04-24T22:40:13.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.600+0000 7feb00c00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feae006be30 0x7feae006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.601+0000 7feb00c00640 1 -- 172.21.15.47:0/2528511795 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feaf408a2b0 con 0x7feb040c81d0 2024-04-24T22:40:13.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.601+0000 7feafbe00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feae006be30 0x7feae006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:13.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.601+0000 7feafbe00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feae006be30 0x7feae006e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb040ca2b0 tx=0x7feaec007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:13.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.605+0000 7feb00c00640 1 -- 172.21.15.47:0/2528511795 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feaf40584e0 con 0x7feb040c81d0 2024-04-24T22:40:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.846+0000 7feb0ac00640 1 -- 172.21.15.47:0/2528511795 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7feb040b1e90 con 0x7feb040c81d0 2024-04-24T22:40:13.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.847+0000 7feb00c00640 1 -- 172.21.15.47:0/2528511795 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7feaf405c7e0 con 0x7feb040c81d0 2024-04-24T22:40:13.849 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T22:40:13.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 -- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feae006be30 msgr2=0x7feae006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feae006be30 0x7feae006e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb040ca2b0 tx=0x7feaec007d70 comp rx=0 tx=0).stop 2024-04-24T22:40:13.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 -- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb040c81d0 msgr2=0x7feb041f0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:13.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb040c81d0 0x7feb041f0190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c3f0 tx=0x7feaf400c8b0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 -- 172.21.15.47:0/2528511795 shutdown_connections 2024-04-24T22:40:13.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feae006be30 0x7feae006e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0415ad80 0x7feb0415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb041549c0 0x7feb04159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 --2- 172.21.15.47:0/2528511795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb040c81d0 0x7feb041f0190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:13.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 -- 172.21.15.47:0/2528511795 >> 172.21.15.47:0/2528511795 conn(0x7feb040c9cb0 msgr2=0x7feb04066ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:13.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 -- 172.21.15.47:0/2528511795 shutdown_connections 2024-04-24T22:40:13.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:13.850+0000 7feb0ac00640 1 -- 172.21.15.47:0/2528511795 wait complete. 2024-04-24T22:40:13.875 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T22:40:13.875 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T22:40:13.875 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-24T22:40:13.876 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T22:40:13.876 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T22:40:13.876 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T22:40:13.876 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T22:40:13.876 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T22:40:13.876 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T22:40:13.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:40:13.876 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-24T22:40:13.928 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T22:40:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T22:40:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T22:40:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42610sec preferred_lft 42610sec 2024-04-24T22:40:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-24T22:40:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-24T22:40:13.930 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T22:40:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-24T22:40:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-24T22:40:13.931 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T22:40:13.931 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:40:13.931 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-24T22:40:13.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-04-24T22:40:13.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-04-24T22:40:13.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-04-24T22:40:13.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T22:40:13.931 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-24T22:40:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:40:14.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:40:14.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:40:14.438 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T22:40:14.507 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-24T22:40:14.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:40:14.511 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-24T22:40:14.576 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-24T22:40:14.576 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T22:40:14.576 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T22:40:14.577 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T22:40:14.577 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T22:40:14.577 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:40:14.578 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-24T22:40:14.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T22:40:14.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T22:40:14.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T22:40:14.578 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-24T22:40:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:40:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:40:15.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:40:15.359 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-24T22:40:15.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:40:15.407 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-04-24T22:40:15.498 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:40:15.498 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-24T22:40:15.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:40:15.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T22:40:15.498 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-24T22:40:15.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:40:15.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:40:15.679 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T22:40:15.679 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:40:15.679 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-24T22:40:15.679 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T22:40:15.679 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T22:40:15.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T22:40:15.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T22:40:15.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T22:40:15.680 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-24T22:40:15.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:40:15.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:40:15.975 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T22:40:15.975 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-24T22:40:15.975 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-24T22:40:15.976 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T22:40:15.976 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-24T22:40:16.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:40:16.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:40:16.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:40:16.179 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T22:40:16.220 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T22:40:16.221 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T22:40:16.221 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T22:40:16.313 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-24T22:40:16.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:40:16.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:40:16.434 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:40:16.435 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T22:40:17.159 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-24T22:40:17.159 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-24T22:40:17.159 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-04-24T22:40:17.159 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with new device syntax: 0@617410c3-1ae2-44fe-8492-d21ee235c3cc.cephfs=/ 2024-04-24T22:40:17.159 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.47:3300/172.21.15.113:3300/172.21.15.113:3301" will pass to kernel 2024-04-24T22:40:17.168 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T22:40:17.168 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-04-24T22:40:17.168 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T22:40:17.168 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T22:40:17.168 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T22:40:17.174 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T22:40:17.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:40:17.174 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T22:40:17.279 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:40:17.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:40:17.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:40:17.438 INFO:teuthology.orchestra.run.smithi047.stdout:client4619 2024-04-24T22:40:17.438 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:40:17.438 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:40:17.496 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:40:17.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:40:17.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:40:17.652 INFO:teuthology.orchestra.run.smithi047.stdout:client4619 2024-04-24T22:40:17.653 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T22:40:17.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T22:40:17.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:40:17.793 INFO:teuthology.orchestra.run.smithi047.stdout:client4619 2024-04-24T22:40:17.794 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/617410c3-1ae2-44fe-8492-d21ee235c3cc.client4619/status') 2024-04-24T22:40:17.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T22:40:17.939 INFO:teuthology.orchestra.run.smithi047.stdout:instance: client.4619 (3)172.21.15.47:0/3986229317 2024-04-24T22:40:17.940 INFO:teuthology.orchestra.run.smithi047.stdout:blocklisted: false 2024-04-24T22:40:17.940 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records in 2024-04-24T22:40:17.940 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records out 2024-04-24T22:40:17.940 INFO:teuthology.orchestra.run.smithi047.stderr:70 bytes copied, 8.3878e-05 s, 835 kB/s 2024-04-24T22:40:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T22:40:17.956 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-04-24T22:40:18.065 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-04-24T22:40:18.066 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:40:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.284+0000 7feede600640 1 Processor -- start 2024-04-24T22:40:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.285+0000 7feede600640 1 -- start start 2024-04-24T22:40:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.286+0000 7feede600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815a680 0x7feed80c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.286+0000 7feede600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feed80c8ad0 0x7feed80c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.286+0000 7feede600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed815ab10 con 0x7feed8154960 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.286+0000 7feede600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7feed815adf0 con 0x7feed80c8ad0 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.286+0000 7feede600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7feed815b0d0 con 0x7feed815a680 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.287+0000 7feecfe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815a680 0x7feed80c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.287+0000 7feed7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feed80c8ad0 0x7feed80c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.287+0000 7feecfe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815a680 0x7feed80c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34280/0 (socket says 172.21.15.47:34280) 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.287+0000 7feecfe00640 1 -- 172.21.15.47:0/1415535182 learned_addr learned my addr 172.21.15.47:0/1415535182 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.287+0000 7feed7400640 1 --1- 172.21.15.47:0/1415535182 >> v1:172.21.15.113:6789/0 conn(0x7feed8154960 0x7feed815e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:59552/0 (socket says 172.21.15.47:59552) 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.287+0000 7feecfe00640 1 -- 172.21.15.47:0/1415535182 >> v2:172.21.15.113:3300/0 conn(0x7feed80c8ad0 msgr2=0x7feed80c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.287+0000 7feecfe00640 1 --2- 172.21.15.47:0/1415535182 >> v2:172.21.15.113:3300/0 conn(0x7feed80c8ad0 0x7feed80c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.287+0000 7feecfe00640 1 -- 172.21.15.47:0/1415535182 >> v1:172.21.15.113:6789/0 conn(0x7feed8154960 legacy=0x7feed815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.287+0000 7feecfe00640 1 -- 172.21.15.47:0/1415535182 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed815b3b0 con 0x7feed815a680 2024-04-24T22:40:18.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.288+0000 7feecfe00640 1 --2- 172.21.15.47:0/1415535182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815a680 0x7feed80c8590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feec0009870 tx=0x7feec002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c69ed414b07262c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.289+0000 7feed6a00640 1 -- 172.21.15.47:0/1415535182 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0037070 con 0x7feed815a680 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.289+0000 7feed6a00640 1 -- 172.21.15.47:0/1415535182 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec0032440 con 0x7feed815a680 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.289+0000 7feed6a00640 1 -- 172.21.15.47:0/1415535182 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0032810 con 0x7feed815a680 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.289+0000 7feede600640 1 -- 172.21.15.47:0/1415535182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815a680 msgr2=0x7feed80c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.289+0000 7feede600640 1 --2- 172.21.15.47:0/1415535182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815a680 0x7feed80c8590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feec0009870 tx=0x7feec002a250 comp rx=0 tx=0).stop 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.290+0000 7feede600640 1 -- 172.21.15.47:0/1415535182 shutdown_connections 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.290+0000 7feede600640 1 --2- 172.21.15.47:0/1415535182 >> v2:172.21.15.113:3300/0 conn(0x7feed80c8ad0 0x7feed80c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.290+0000 7feede600640 1 --2- 172.21.15.47:0/1415535182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815a680 0x7feed80c8590 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.290+0000 7feede600640 1 -- 172.21.15.47:0/1415535182 >> 172.21.15.47:0/1415535182 conn(0x7feed806e580 msgr2=0x7feed806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.290+0000 7feede600640 1 -- 172.21.15.47:0/1415535182 shutdown_connections 2024-04-24T22:40:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.290+0000 7feede600640 1 -- 172.21.15.47:0/1415535182 wait complete. 2024-04-24T22:40:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.292+0000 7feede600640 1 Processor -- start 2024-04-24T22:40:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.292+0000 7feede600640 1 -- start start 2024-04-24T22:40:18.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.293+0000 7feede600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8ad0 0x7feed8150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.293+0000 7feede600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed8154960 0x7feed815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.293+0000 7feede600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815cdb0 0x7feed814b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.293+0000 7feede600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feed815ea00 con 0x7feed80c8ad0 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.293+0000 7feede600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feed8154d40 con 0x7feed8154960 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.293+0000 7feede600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7feed815eb80 con 0x7feed815cdb0 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.294+0000 7feed7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8ad0 0x7feed8150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.294+0000 7feecfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed8154960 0x7feed815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.294+0000 7feecfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed8154960 0x7feed815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37768/0 (socket says 172.21.15.47:37768) 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.294+0000 7feecfe00640 1 -- 172.21.15.47:0/2065571022 learned_addr learned my addr 172.21.15.47:0/2065571022 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.294+0000 7feed7e00640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815cdb0 0x7feed814b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.294+0000 7feecfe00640 1 -- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815cdb0 msgr2=0x7feed814b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.294+0000 7feecfe00640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815cdb0 0x7feed814b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.294+0000 7feecfe00640 1 -- 172.21.15.47:0/2065571022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8ad0 msgr2=0x7feed8150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.294+0000 7feecfe00640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8ad0 0x7feed8150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.294+0000 7feecfe00640 1 -- 172.21.15.47:0/2065571022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed8152940 con 0x7feed8154960 2024-04-24T22:40:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.295+0000 7feecfe00640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed8154960 0x7feed815d4c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feec002a290 tx=0x7feec00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.295+0000 7feecf400640 1 -- 172.21.15.47:0/2065571022 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0037040 con 0x7feed8154960 2024-04-24T22:40:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.295+0000 7feecf400640 1 -- 172.21.15.47:0/2065571022 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec0042070 con 0x7feed8154960 2024-04-24T22:40:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.295+0000 7feecf400640 1 -- 172.21.15.47:0/2065571022 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0030880 con 0x7feed8154960 2024-04-24T22:40:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.295+0000 7feede600640 1 -- 172.21.15.47:0/2065571022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed80c3130 con 0x7feed8154960 2024-04-24T22:40:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.295+0000 7feede600640 1 -- 172.21.15.47:0/2065571022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed80c6c90 con 0x7feed8154960 2024-04-24T22:40:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.296+0000 7feede600640 1 -- 172.21.15.47:0/2065571022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed8073b90 con 0x7feed8154960 2024-04-24T22:40:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.297+0000 7feecf400640 1 -- 172.21.15.47:0/2065571022 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feec0030a20 con 0x7feed8154960 2024-04-24T22:40:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.298+0000 7feecf400640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feeb406be30 0x7feeb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.298+0000 7feecf400640 1 -- 172.21.15.47:0/2065571022 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feec00ab590 con 0x7feed8154960 2024-04-24T22:40:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.298+0000 7feed7400640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feeb406be30 0x7feeb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.299+0000 7feed7400640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feeb406be30 0x7feeb406e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feec8007fd0 tx=0x7feec8007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.302+0000 7feecf400640 1 -- 172.21.15.47:0/2065571022 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feec00798e0 con 0x7feed8154960 2024-04-24T22:40:18.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.510+0000 7feede600640 1 -- 172.21.15.47:0/2065571022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feed80c76d0 con 0x7feed8154960 2024-04-24T22:40:18.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.511+0000 7feecf400640 1 -- 172.21.15.47:0/2065571022 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16593 (secure 0 0 0) 0x7feec007dbe0 con 0x7feed8154960 2024-04-24T22:40:18.514 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:18.514 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:38.692613+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:40:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 -- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feeb406be30 msgr2=0x7feeb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feeb406be30 0x7feeb406e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feec8007fd0 tx=0x7feec8007e80 comp rx=0 tx=0).stop 2024-04-24T22:40:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 -- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed8154960 msgr2=0x7feed815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed8154960 0x7feed815d4c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feec002a290 tx=0x7feec00046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 -- 172.21.15.47:0/2065571022 shutdown_connections 2024-04-24T22:40:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feeb406be30 0x7feeb406e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feed815cdb0 0x7feed814b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feed8154960 0x7feed815d4c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 --2- 172.21.15.47:0/2065571022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feed80c8ad0 0x7feed8150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 -- 172.21.15.47:0/2065571022 >> 172.21.15.47:0/2065571022 conn(0x7feed806e580 msgr2=0x7feed80727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 -- 172.21.15.47:0/2065571022 shutdown_connections 2024-04-24T22:40:18.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.514+0000 7feede600640 1 -- 172.21.15.47:0/2065571022 wait complete. 2024-04-24T22:40:18.537 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:30.711538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:34.803185+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:36.184123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:40:18.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:40:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.764+0000 7f2886a00640 1 Processor -- start 2024-04-24T22:40:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.765+0000 7f2886a00640 1 -- start start 2024-04-24T22:40:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.766+0000 7f2886a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28801528e0 0x7f288015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.766+0000 7f2886a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2880071a10 0x7f28800c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.766+0000 7f2886a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288006b890 con 0x7f2880073700 2024-04-24T22:40:18.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.766+0000 7f2886a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f288006cb60 con 0x7f28801528e0 2024-04-24T22:40:18.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.766+0000 7f2886a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2880073290 con 0x7f2880071a10 2024-04-24T22:40:18.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.767+0000 7f287fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28801528e0 0x7f288015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.767+0000 7f287f400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2880073700 0x7f28800c7640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56500/0 (socket says 172.21.15.47:56500) 2024-04-24T22:40:18.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.767+0000 7f287f400640 1 -- 172.21.15.47:0/2536126045 learned_addr learned my addr 172.21.15.47:0/2536126045 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:18.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.767+0000 7f2884a00640 1 --2- 172.21.15.47:0/2536126045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2880071a10 0x7f28800c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.767+0000 7f287fe00640 1 -- 172.21.15.47:0/2536126045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2880071a10 msgr2=0x7f28800c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.767+0000 7f287fe00640 1 --2- 172.21.15.47:0/2536126045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2880071a10 0x7f28800c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.767+0000 7f287fe00640 1 -- 172.21.15.47:0/2536126045 >> v1:172.21.15.47:6789/0 conn(0x7f2880073700 legacy=0x7f28800c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.768+0000 7f287fe00640 1 -- 172.21.15.47:0/2536126045 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288015c8f0 con 0x7f28801528e0 2024-04-24T22:40:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.768+0000 7f287ea00640 1 -- 172.21.15.47:0/2536126045 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829638609 0 0) 0x7f288006b890 con 0x7f2880073700 2024-04-24T22:40:18.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.769+0000 7f287fe00640 1 --2- 172.21.15.47:0/2536126045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28801528e0 0x7f288015c3b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f288015cac0 tx=0x7f287002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c46a2c991b92d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.769+0000 7f287ea00640 1 -- 172.21.15.47:0/2536126045 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870007740 con 0x7f28801528e0 2024-04-24T22:40:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.769+0000 7f287ea00640 1 -- 172.21.15.47:0/2536126045 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28700078e0 con 0x7f28801528e0 2024-04-24T22:40:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.769+0000 7f287ea00640 1 -- 172.21.15.47:0/2536126045 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870007cb0 con 0x7f28801528e0 2024-04-24T22:40:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.770+0000 7f2886a00640 1 -- 172.21.15.47:0/2536126045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28801528e0 msgr2=0x7f288015c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.770+0000 7f2886a00640 1 --2- 172.21.15.47:0/2536126045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28801528e0 0x7f288015c3b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f288015cac0 tx=0x7f287002a6d0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.770+0000 7f2886a00640 1 -- 172.21.15.47:0/2536126045 shutdown_connections 2024-04-24T22:40:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.770+0000 7f2886a00640 1 --2- 172.21.15.47:0/2536126045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2880071a10 0x7f28800c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.770+0000 7f2886a00640 1 --2- 172.21.15.47:0/2536126045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28801528e0 0x7f288015c3b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.770+0000 7f2886a00640 1 -- 172.21.15.47:0/2536126045 >> 172.21.15.47:0/2536126045 conn(0x7f288006f520 msgr2=0x7f288006f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.770+0000 7f2886a00640 1 -- 172.21.15.47:0/2536126045 shutdown_connections 2024-04-24T22:40:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.770+0000 7f2886a00640 1 -- 172.21.15.47:0/2536126045 wait complete. 2024-04-24T22:40:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.771+0000 7f2886a00640 1 Processor -- start 2024-04-24T22:40:18.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.772+0000 7f2886a00640 1 -- start start 2024-04-24T22:40:18.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.772+0000 7f2886a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2880071a10 0x7f28800c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.772+0000 7f2886a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2880073700 0x7f28800c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f2886a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28801528e0 0x7f288015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f2886a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f288006bb30 con 0x7f2880071a10 2024-04-24T22:40:18.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f2886a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f288006c880 con 0x7f28801528e0 2024-04-24T22:40:18.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f2886a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f28800c7f40 con 0x7f2880073700 2024-04-24T22:40:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f287fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2880071a10 0x7f28800c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f287fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2880071a10 0x7f28800c51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60506/0 (socket says 172.21.15.47:60506) 2024-04-24T22:40:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f287fe00640 1 -- 172.21.15.47:0/2298243620 learned_addr learned my addr 172.21.15.47:0/2298243620 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f287f400640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2880073700 0x7f28800c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f2884a00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28801528e0 0x7f288015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f287fe00640 1 -- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2880073700 msgr2=0x7f28800c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f287fe00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2880073700 0x7f28800c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f287fe00640 1 -- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28801528e0 msgr2=0x7f288015b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f287fe00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28801528e0 0x7f288015b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.773+0000 7f287fe00640 1 -- 172.21.15.47:0/2298243620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28800cc860 con 0x7f2880071a10 2024-04-24T22:40:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.774+0000 7f287fe00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2880071a10 0x7f28800c51c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2870009e80 tx=0x7f2870004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.774+0000 7f2877e00640 1 -- 172.21.15.47:0/2298243620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870040030 con 0x7f2880071a10 2024-04-24T22:40:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.774+0000 7f2877e00640 1 -- 172.21.15.47:0/2298243620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2870031a40 con 0x7f2880071a10 2024-04-24T22:40:18.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.774+0000 7f2877e00640 1 -- 172.21.15.47:0/2298243620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870009450 con 0x7f2880071a10 2024-04-24T22:40:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.774+0000 7f2886a00640 1 -- 172.21.15.47:0/2298243620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28800ccb00 con 0x7f2880071a10 2024-04-24T22:40:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.774+0000 7f2886a00640 1 -- 172.21.15.47:0/2298243620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28800cd0c0 con 0x7f2880071a10 2024-04-24T22:40:18.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.776+0000 7f2886a00640 1 -- 172.21.15.47:0/2298243620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2880073b90 con 0x7f2880071a10 2024-04-24T22:40:18.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.776+0000 7f2877e00640 1 -- 172.21.15.47:0/2298243620 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f287003b410 con 0x7f2880071a10 2024-04-24T22:40:18.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.777+0000 7f2877e00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f285c06be30 0x7f285c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:18.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.777+0000 7f2877e00640 1 -- 172.21.15.47:0/2298243620 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f28700abd90 con 0x7f2880071a10 2024-04-24T22:40:18.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.778+0000 7f287f400640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f285c06be30 0x7f285c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:18.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.778+0000 7f287f400640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f285c06be30 0x7f285c06e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2868007f70 tx=0x7f2868007f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:18.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.782+0000 7f2877e00640 1 -- 172.21.15.47:0/2298243620 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2870079fc0 con 0x7f2880071a10 2024-04-24T22:40:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.989+0000 7f2886a00640 1 -- 172.21.15.47:0/2298243620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28800c6d10 con 0x7f2880071a10 2024-04-24T22:40:18.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.990+0000 7f2877e00640 1 -- 172.21.15.47:0/2298243620 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f287007e2c0 con 0x7f2880071a10 2024-04-24T22:40:18.993 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-24T22:40:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.993+0000 7f2886a00640 1 -- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f285c06be30 msgr2=0x7f285c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.993+0000 7f2886a00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f285c06be30 0x7f285c06e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2868007f70 tx=0x7f2868007f00 comp rx=0 tx=0).stop 2024-04-24T22:40:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.993+0000 7f2886a00640 1 -- 172.21.15.47:0/2298243620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2880071a10 msgr2=0x7f28800c51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.993+0000 7f2886a00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2880071a10 0x7f28800c51c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2870009e80 tx=0x7f2870004a40 comp rx=0 tx=0).stop 2024-04-24T22:40:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.993+0000 7f2886a00640 1 -- 172.21.15.47:0/2298243620 shutdown_connections 2024-04-24T22:40:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.993+0000 7f2886a00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f285c06be30 0x7f285c06e2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.993+0000 7f2886a00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28801528e0 0x7f288015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.993+0000 7f2886a00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2880073700 0x7f28800c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.993+0000 7f2886a00640 1 --2- 172.21.15.47:0/2298243620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2880071a10 0x7f28800c51c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.993+0000 7f2886a00640 1 -- 172.21.15.47:0/2298243620 >> 172.21.15.47:0/2298243620 conn(0x7f288006f520 msgr2=0x7f28800b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.994+0000 7f2886a00640 1 -- 172.21.15.47:0/2298243620 shutdown_connections 2024-04-24T22:40:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:18.994+0000 7f2886a00640 1 -- 172.21.15.47:0/2298243620 wait complete. 2024-04-24T22:40:19.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:40:19.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.229+0000 7f9214e00640 1 Processor -- start 2024-04-24T22:40:19.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.229+0000 7f9214e00640 1 -- start start 2024-04-24T22:40:19.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.231+0000 7f9214e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9210073040 0x7f92101411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.231+0000 7f9214e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f921015c3d0 con 0x7f9210072c60 2024-04-24T22:40:19.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.231+0000 7f9214e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f921015c6b0 con 0x7f921006bb90 2024-04-24T22:40:19.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.231+0000 7f9214e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9210158bf0 con 0x7f9210073040 2024-04-24T22:40:19.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.231+0000 7f920f400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9210072c60 0x7f92100c2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56502/0 (socket says 172.21.15.47:56502) 2024-04-24T22:40:19.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.231+0000 7f920f400640 1 -- 172.21.15.47:0/1292282395 learned_addr learned my addr 172.21.15.47:0/1292282395 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:19.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.231+0000 7f920ea00640 1 --2- 172.21.15.47:0/1292282395 >> v2:172.21.15.47:3300/0 conn(0x7f9210073040 0x7f92101411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.231+0000 7f920e000640 1 -- 172.21.15.47:0/1292282395 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737165410 0 0) 0x7f921015c3d0 con 0x7f9210072c60 2024-04-24T22:40:19.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.232+0000 7f920e000640 1 -- 172.21.15.47:0/1292282395 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91f0003520 con 0x7f9210072c60 2024-04-24T22:40:19.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.232+0000 7f920e000640 1 -- 172.21.15.47:0/1292282395 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3087126887 0 0) 0x7f921015c6b0 con 0x7f921006bb90 2024-04-24T22:40:19.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.232+0000 7f920e000640 1 -- 172.21.15.47:0/1292282395 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f921015c3d0 con 0x7f921006bb90 2024-04-24T22:40:19.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.232+0000 7f920ea00640 1 -- 172.21.15.47:0/1292282395 >> v1:172.21.15.113:6789/0 conn(0x7f921006bb90 legacy=0x7f921015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.232+0000 7f920ea00640 1 -- 172.21.15.47:0/1292282395 >> v1:172.21.15.47:6789/0 conn(0x7f9210072c60 legacy=0x7f92100c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.232+0000 7f920ea00640 1 -- 172.21.15.47:0/1292282395 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92101585a0 con 0x7f9210073040 2024-04-24T22:40:19.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.233+0000 7f920ea00640 1 --2- 172.21.15.47:0/1292282395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9210073040 0x7f92101411f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009d90 tx=0x7f91f802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84fa4d1e85fd48a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.233+0000 7f920e000640 1 -- 172.21.15.47:0/1292282395 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8007740 con 0x7f9210073040 2024-04-24T22:40:19.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.233+0000 7f920e000640 1 -- 172.21.15.47:0/1292282395 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f80078e0 con 0x7f9210073040 2024-04-24T22:40:19.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.233+0000 7f920e000640 1 -- 172.21.15.47:0/1292282395 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8007c90 con 0x7f9210073040 2024-04-24T22:40:19.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.234+0000 7f9214e00640 1 -- 172.21.15.47:0/1292282395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9210073040 msgr2=0x7f92101411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.234+0000 7f9214e00640 1 --2- 172.21.15.47:0/1292282395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9210073040 0x7f92101411f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009d90 tx=0x7f91f802a6f0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.234+0000 7f9214e00640 1 -- 172.21.15.47:0/1292282395 shutdown_connections 2024-04-24T22:40:19.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.234+0000 7f9214e00640 1 --2- 172.21.15.47:0/1292282395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9210073040 0x7f92101411f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.234+0000 7f9214e00640 1 -- 172.21.15.47:0/1292282395 >> 172.21.15.47:0/1292282395 conn(0x7f92100cb900 msgr2=0x7f92100cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:19.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.234+0000 7f9214e00640 1 -- 172.21.15.47:0/1292282395 shutdown_connections 2024-04-24T22:40:19.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.234+0000 7f9214e00640 1 -- 172.21.15.47:0/1292282395 wait complete. 2024-04-24T22:40:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.236+0000 7f9214e00640 1 Processor -- start 2024-04-24T22:40:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.236+0000 7f9214e00640 1 -- start start 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f9214e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f921006bb90 0x7f9210071c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f9214e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9210072c60 0x7f92101e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f9214e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9210073040 0x7f92101e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f9214e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9210159080 con 0x7f921006bb90 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f9214e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9210158270 con 0x7f9210072c60 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f9214e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9210159200 con 0x7f9210073040 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f920f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f921006bb90 0x7f9210071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f920ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9210072c60 0x7f92101e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f920ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9210072c60 0x7f92101e58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37792/0 (socket says 172.21.15.47:37792) 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f920ea00640 1 -- 172.21.15.47:0/2113552671 learned_addr learned my addr 172.21.15.47:0/2113552671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:19.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.237+0000 7f920fe00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9210073040 0x7f92101e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.238+0000 7f920ea00640 1 -- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9210073040 msgr2=0x7f92101e5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.238+0000 7f920ea00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9210073040 0x7f92101e5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.238+0000 7f920ea00640 1 -- 172.21.15.47:0/2113552671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f921006bb90 msgr2=0x7f9210071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.238+0000 7f920ea00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f921006bb90 0x7f9210071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.238+0000 7f920ea00640 1 -- 172.21.15.47:0/2113552671 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92100cb670 con 0x7f9210072c60 2024-04-24T22:40:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.238+0000 7f920fe00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9210073040 0x7f92101e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.238+0000 7f920ea00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9210072c60 0x7f92101e58d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009eb0 tx=0x7f91f8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.238+0000 7f9207400640 1 -- 172.21.15.47:0/2113552671 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8040070 con 0x7f9210072c60 2024-04-24T22:40:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.239+0000 7f9207400640 1 -- 172.21.15.47:0/2113552671 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f8031c40 con 0x7f9210072c60 2024-04-24T22:40:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.239+0000 7f9207400640 1 -- 172.21.15.47:0/2113552671 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8009450 con 0x7f9210072c60 2024-04-24T22:40:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.239+0000 7f9214e00640 1 -- 172.21.15.47:0/2113552671 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92100c9c10 con 0x7f9210072c60 2024-04-24T22:40:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.239+0000 7f9214e00640 1 -- 172.21.15.47:0/2113552671 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92101f44b0 con 0x7f9210072c60 2024-04-24T22:40:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.240+0000 7f9214e00640 1 -- 172.21.15.47:0/2113552671 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9210068d50 con 0x7f9210072c60 2024-04-24T22:40:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.241+0000 7f9207400640 1 -- 172.21.15.47:0/2113552671 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91f8005d10 con 0x7f9210072c60 2024-04-24T22:40:19.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.242+0000 7f9207400640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f91ec06be30 0x7f91ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.242+0000 7f9207400640 1 -- 172.21.15.47:0/2113552671 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f91f80abb20 con 0x7f9210072c60 2024-04-24T22:40:19.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.242+0000 7f920f400640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f91ec06be30 0x7f91ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.243+0000 7f920f400640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f91ec06be30 0x7f91ec06e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9200000f60 tx=0x7f9200007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.247+0000 7f9207400640 1 -- 172.21.15.47:0/2113552671 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91f804b020 con 0x7f9210072c60 2024-04-24T22:40:19.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.448+0000 7f9214e00640 1 -- 172.21.15.47:0/2113552671 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f921004f5e0 con 0x7f9210072c60 2024-04-24T22:40:19.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.449+0000 7f9207400640 1 -- 172.21.15.47:0/2113552671 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f91f8079d50 con 0x7f9210072c60 2024-04-24T22:40:19.451 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-24T22:40:19.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.451+0000 7f9214e00640 1 -- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f91ec06be30 msgr2=0x7f91ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.451+0000 7f9214e00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f91ec06be30 0x7f91ec06e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9200000f60 tx=0x7f9200007d70 comp rx=0 tx=0).stop 2024-04-24T22:40:19.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.451+0000 7f9214e00640 1 -- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9210072c60 msgr2=0x7f92101e58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.451+0000 7f9214e00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9210072c60 0x7f92101e58d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009eb0 tx=0x7f91f8004a40 comp rx=0 tx=0).stop 2024-04-24T22:40:19.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.451+0000 7f9214e00640 1 -- 172.21.15.47:0/2113552671 shutdown_connections 2024-04-24T22:40:19.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.451+0000 7f9214e00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f91ec06be30 0x7f91ec06e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.451+0000 7f9214e00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9210073040 0x7f92101e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.451+0000 7f9214e00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9210072c60 0x7f92101e58d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.451+0000 7f9214e00640 1 --2- 172.21.15.47:0/2113552671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f921006bb90 0x7f9210071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.451+0000 7f9214e00640 1 -- 172.21.15.47:0/2113552671 >> 172.21.15.47:0/2113552671 conn(0x7f92100cb900 msgr2=0x7f921015b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:19.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.452+0000 7f9214e00640 1 -- 172.21.15.47:0/2113552671 shutdown_connections 2024-04-24T22:40:19.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.452+0000 7f9214e00640 1 -- 172.21.15.47:0/2113552671 wait complete. 2024-04-24T22:40:19.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:40:19.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.690+0000 7f3eebc00640 1 Processor -- start 2024-04-24T22:40:19.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.691+0000 7f3eebc00640 1 -- start start 2024-04-24T22:40:19.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.693+0000 7f3eebc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ee415a680 0x7f3ee40c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.693+0000 7f3eebc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 0x7f3ee40c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.693+0000 7f3eebc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ee415ab10 con 0x7f3ee4154960 2024-04-24T22:40:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.693+0000 7f3eebc00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3ee415adf0 con 0x7f3ee415a680 2024-04-24T22:40:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.693+0000 7f3eebc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee415b0d0 con 0x7f3ee40c8ad0 2024-04-24T22:40:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.693+0000 7f3ee9200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3ee4154960 0x7f3ee415e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:59560/0 (socket says 172.21.15.47:59560) 2024-04-24T22:40:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.693+0000 7f3ee9200640 1 -- 172.21.15.47:0/649640377 learned_addr learned my addr 172.21.15.47:0/649640377 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.693+0000 7f3edbe00640 1 --2- 172.21.15.47:0/649640377 >> v2:172.21.15.47:3300/0 conn(0x7f3ee415a680 0x7f3ee40c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.693+0000 7f3ee9c00640 1 --2- 172.21.15.47:0/649640377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 0x7f3ee40c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.694+0000 7f3edbe00640 1 -- 172.21.15.47:0/649640377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 msgr2=0x7f3ee40c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:19.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.694+0000 7f3edbe00640 1 --2- 172.21.15.47:0/649640377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 0x7f3ee40c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.694+0000 7f3edbe00640 1 -- 172.21.15.47:0/649640377 >> v1:172.21.15.113:6789/0 conn(0x7f3ee4154960 legacy=0x7f3ee415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.694+0000 7f3ee3e00640 1 -- 172.21.15.47:0/649640377 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3227019421 0 0) 0x7f3ee415ab10 con 0x7f3ee4154960 2024-04-24T22:40:19.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.694+0000 7f3edbe00640 1 -- 172.21.15.47:0/649640377 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee415b3b0 con 0x7f3ee415a680 2024-04-24T22:40:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.695+0000 7f3edbe00640 1 --2- 172.21.15.47:0/649640377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee415a680 0x7f3ee40c8590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f064ce2912c7f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.695+0000 7f3ee3e00640 1 -- 172.21.15.47:0/649640377 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc037070 con 0x7f3ee415a680 2024-04-24T22:40:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.695+0000 7f3ee3e00640 1 -- 172.21.15.47:0/649640377 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc032440 con 0x7f3ee415a680 2024-04-24T22:40:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.695+0000 7f3ee3e00640 1 -- 172.21.15.47:0/649640377 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc032810 con 0x7f3ee415a680 2024-04-24T22:40:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.696+0000 7f3eebc00640 1 -- 172.21.15.47:0/649640377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee415a680 msgr2=0x7f3ee40c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.696+0000 7f3eebc00640 1 --2- 172.21.15.47:0/649640377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee415a680 0x7f3ee40c8590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc02a180 comp rx=0 tx=0).stop 2024-04-24T22:40:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.696+0000 7f3eebc00640 1 -- 172.21.15.47:0/649640377 shutdown_connections 2024-04-24T22:40:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.696+0000 7f3eebc00640 1 --2- 172.21.15.47:0/649640377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 0x7f3ee40c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.696+0000 7f3eebc00640 1 --2- 172.21.15.47:0/649640377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee415a680 0x7f3ee40c8590 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.696+0000 7f3eebc00640 1 -- 172.21.15.47:0/649640377 >> 172.21.15.47:0/649640377 conn(0x7f3ee406e580 msgr2=0x7f3ee406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.696+0000 7f3eebc00640 1 -- 172.21.15.47:0/649640377 shutdown_connections 2024-04-24T22:40:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.696+0000 7f3eebc00640 1 -- 172.21.15.47:0/649640377 wait complete. 2024-04-24T22:40:19.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.698+0000 7f3eebc00640 1 Processor -- start 2024-04-24T22:40:19.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.698+0000 7f3eebc00640 1 -- start start 2024-04-24T22:40:19.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3eebc00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 0x7f3ee415caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3eebc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4154960 0x7f3ee41f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3eebc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee41f0b80 0x7f3ee4146ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3eebc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee415ea00 con 0x7f3ee4154960 2024-04-24T22:40:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3eebc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee4154d40 con 0x7f3ee41f0b80 2024-04-24T22:40:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3eebc00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee415eb80 con 0x7f3ee40c8ad0 2024-04-24T22:40:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3edbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4154960 0x7f3ee41f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3edbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4154960 0x7f3ee41f0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60526/0 (socket says 172.21.15.47:60526) 2024-04-24T22:40:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3edbe00640 1 -- 172.21.15.47:0/3475450028 learned_addr learned my addr 172.21.15.47:0/3475450028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3ee9c00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee41f0b80 0x7f3ee4146ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.699+0000 7f3ee9200640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 0x7f3ee415caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3edbe00640 1 -- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 msgr2=0x7f3ee415caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3edbe00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 0x7f3ee415caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3edbe00640 1 -- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee41f0b80 msgr2=0x7f3ee4146ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3edbe00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee41f0b80 0x7f3ee4146ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3edbe00640 1 -- 172.21.15.47:0/3475450028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee41f3e80 con 0x7f3ee4154960 2024-04-24T22:40:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3ee9c00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee41f0b80 0x7f3ee4146ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3ee9200640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 0x7f3ee415caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3edbe00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4154960 0x7f3ee41f0640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3ee1600640 1 -- 172.21.15.47:0/3475450028 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc037050 con 0x7f3ee4154960 2024-04-24T22:40:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3ee1600640 1 -- 172.21.15.47:0/3475450028 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc038040 con 0x7f3ee4154960 2024-04-24T22:40:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3ee1600640 1 -- 172.21.15.47:0/3475450028 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc004490 con 0x7f3ee4154960 2024-04-24T22:40:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.700+0000 7f3eebc00640 1 -- 172.21.15.47:0/3475450028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee4154ff0 con 0x7f3ee4154960 2024-04-24T22:40:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.701+0000 7f3eebc00640 1 -- 172.21.15.47:0/3475450028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee415c490 con 0x7f3ee4154960 2024-04-24T22:40:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.702+0000 7f3eebc00640 1 -- 172.21.15.47:0/3475450028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee4154510 con 0x7f3ee4154960 2024-04-24T22:40:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.703+0000 7f3ee1600640 1 -- 172.21.15.47:0/3475450028 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ecc03c6d0 con 0x7f3ee4154960 2024-04-24T22:40:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.704+0000 7f3ee1600640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ec006be30 0x7f3ec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.704+0000 7f3ee1600640 1 -- 172.21.15.47:0/3475450028 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3ecc0ab650 con 0x7f3ee4154960 2024-04-24T22:40:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.704+0000 7f3ee9200640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ec006be30 0x7f3ec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.705+0000 7f3ee9200640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ec006be30 0x7f3ec006e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4007f70 tx=0x7f3ed4007f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:19.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.708+0000 7f3ee1600640 1 -- 172.21.15.47:0/3475450028 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ee4154510 con 0x7f3ee4154960 2024-04-24T22:40:19.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.913+0000 7f3eebc00640 1 -- 172.21.15.47:0/3475450028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3ee415ab10 con 0x7f3ee4154960 2024-04-24T22:40:19.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.913+0000 7f3ee1600640 1 -- 172.21.15.47:0/3475450028 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f3ee415ab10 con 0x7f3ee4154960 2024-04-24T22:40:19.915 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 -- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ec006be30 msgr2=0x7f3ec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ec006be30 0x7f3ec006e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4007f70 tx=0x7f3ed4007f00 comp rx=0 tx=0).stop 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 -- 172.21.15.47:0/3475450028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4154960 msgr2=0x7f3ee41f0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4154960 0x7f3ee41f0640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 -- 172.21.15.47:0/3475450028 shutdown_connections 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ec006be30 0x7f3ec006e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ee41f0b80 0x7f3ee4146ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ee4154960 0x7f3ee41f0640 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 --2- 172.21.15.47:0/3475450028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ee40c8ad0 0x7f3ee415caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 -- 172.21.15.47:0/3475450028 >> 172.21.15.47:0/3475450028 conn(0x7f3ee406e580 msgr2=0x7f3ee406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 -- 172.21.15.47:0/3475450028 shutdown_connections 2024-04-24T22:40:19.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:19.916+0000 7f3eebc00640 1 -- 172.21.15.47:0/3475450028 wait complete. 2024-04-24T22:40:19.937 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-04-24T22:40:19.938 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:40:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.147+0000 7fd593200640 1 Processor -- start 2024-04-24T22:40:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.147+0000 7fd593200640 1 -- start start 2024-04-24T22:40:20.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.149+0000 7fd593200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8f10 0x7fd594066f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.149+0000 7fd593200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd59415a730 con 0x7fd5940c8b30 2024-04-24T22:40:20.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.149+0000 7fd593200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd59415a9b0 con 0x7fd594154c30 2024-04-24T22:40:20.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.149+0000 7fd593200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd59406c730 con 0x7fd5940c8f10 2024-04-24T22:40:20.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.149+0000 7fd591e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd5940c8b30 0x7fd594154520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56506/0 (socket says 172.21.15.47:56506) 2024-04-24T22:40:20.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.149+0000 7fd591e00640 1 -- 172.21.15.47:0/1162910752 learned_addr learned my addr 172.21.15.47:0/1162910752 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:20.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.149+0000 7fd592800640 1 --2- 172.21.15.47:0/1162910752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8f10 0x7fd594066f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.150+0000 7fd592800640 1 -- 172.21.15.47:0/1162910752 >> v1:172.21.15.113:6789/0 conn(0x7fd594154c30 legacy=0x7fd59415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.150+0000 7fd591400640 1 -- 172.21.15.47:0/1162910752 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1862053969 0 0) 0x7fd59415a9b0 con 0x7fd594154c30 2024-04-24T22:40:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.150+0000 7fd592800640 1 -- 172.21.15.47:0/1162910752 >> v1:172.21.15.47:6789/0 conn(0x7fd5940c8b30 legacy=0x7fd594154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.150+0000 7fd592800640 1 -- 172.21.15.47:0/1162910752 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59415ac90 con 0x7fd5940c8f10 2024-04-24T22:40:20.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.151+0000 7fd592800640 1 --2- 172.21.15.47:0/1162910752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8f10 0x7fd594066f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0097d0 tx=0x7fd58c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68744c01b4847052 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.151+0000 7fd591400640 1 -- 172.21.15.47:0/1162910752 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c037070 con 0x7fd5940c8f10 2024-04-24T22:40:20.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.151+0000 7fd591400640 1 -- 172.21.15.47:0/1162910752 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58c02ad90 con 0x7fd5940c8f10 2024-04-24T22:40:20.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.151+0000 7fd591400640 1 -- 172.21.15.47:0/1162910752 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c032660 con 0x7fd5940c8f10 2024-04-24T22:40:20.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.152+0000 7fd593200640 1 -- 172.21.15.47:0/1162910752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8f10 msgr2=0x7fd594066f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.152+0000 7fd593200640 1 --2- 172.21.15.47:0/1162910752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8f10 0x7fd594066f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0097d0 tx=0x7fd58c02a100 comp rx=0 tx=0).stop 2024-04-24T22:40:20.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.152+0000 7fd593200640 1 -- 172.21.15.47:0/1162910752 shutdown_connections 2024-04-24T22:40:20.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.152+0000 7fd593200640 1 --2- 172.21.15.47:0/1162910752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8f10 0x7fd594066f80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.152+0000 7fd593200640 1 -- 172.21.15.47:0/1162910752 >> 172.21.15.47:0/1162910752 conn(0x7fd5940c9460 msgr2=0x7fd5940c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:20.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.152+0000 7fd593200640 1 -- 172.21.15.47:0/1162910752 shutdown_connections 2024-04-24T22:40:20.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.152+0000 7fd593200640 1 -- 172.21.15.47:0/1162910752 wait complete. 2024-04-24T22:40:20.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.154+0000 7fd593200640 1 Processor -- start 2024-04-24T22:40:20.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.154+0000 7fd593200640 1 -- start start 2024-04-24T22:40:20.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.155+0000 7fd593200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8b30 0x7fd5940c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.155+0000 7fd593200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5940c8f10 0x7fd5940cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.155+0000 7fd593200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd59415db70 0x7fd59415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.155+0000 7fd593200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd59415b530 con 0x7fd5940c8f10 2024-04-24T22:40:20.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.155+0000 7fd593200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd59415b250 con 0x7fd59415db70 2024-04-24T22:40:20.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.155+0000 7fd593200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd59415e8c0 con 0x7fd5940c8b30 2024-04-24T22:40:20.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.155+0000 7fd591e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8b30 0x7fd5940c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.155+0000 7fd589400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5940c8f10 0x7fd5940cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.156+0000 7fd591e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8b30 0x7fd5940c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34346/0 (socket says 172.21.15.47:34346) 2024-04-24T22:40:20.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.156+0000 7fd592800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd59415db70 0x7fd59415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.156+0000 7fd591e00640 1 -- 172.21.15.47:0/4113282565 learned_addr learned my addr 172.21.15.47:0/4113282565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:20.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.156+0000 7fd591e00640 1 -- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd59415db70 msgr2=0x7fd59415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.156+0000 7fd591e00640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd59415db70 0x7fd59415c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.156+0000 7fd591e00640 1 -- 172.21.15.47:0/4113282565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5940c8f10 msgr2=0x7fd5940cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.156+0000 7fd591e00640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5940c8f10 0x7fd5940cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.156+0000 7fd591e00640 1 -- 172.21.15.47:0/4113282565 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5940cc290 con 0x7fd5940c8b30 2024-04-24T22:40:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.156+0000 7fd591e00640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8b30 0x7fd5940c47f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd58400cc10 tx=0x7fd58400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.156+0000 7fd58aa00640 1 -- 172.21.15.47:0/4113282565 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584018070 con 0x7fd5940c8b30 2024-04-24T22:40:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.157+0000 7fd58aa00640 1 -- 172.21.15.47:0/4113282565 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd584004d80 con 0x7fd5940c8b30 2024-04-24T22:40:20.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.157+0000 7fd58aa00640 1 -- 172.21.15.47:0/4113282565 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58400ad80 con 0x7fd5940c8b30 2024-04-24T22:40:20.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.157+0000 7fd593200640 1 -- 172.21.15.47:0/4113282565 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5940cc530 con 0x7fd5940c8b30 2024-04-24T22:40:20.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.157+0000 7fd593200640 1 -- 172.21.15.47:0/4113282565 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5940cad20 con 0x7fd5940c8b30 2024-04-24T22:40:20.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.158+0000 7fd593200640 1 -- 172.21.15.47:0/4113282565 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd59404f5e0 con 0x7fd5940c8b30 2024-04-24T22:40:20.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.159+0000 7fd58aa00640 1 -- 172.21.15.47:0/4113282565 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd58401b070 con 0x7fd5940c8b30 2024-04-24T22:40:20.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.160+0000 7fd58aa00640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd57006be30 0x7fd57006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.160+0000 7fd58aa00640 1 -- 172.21.15.47:0/4113282565 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd58408a860 con 0x7fd5940c8b30 2024-04-24T22:40:20.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.160+0000 7fd589400640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd57006be30 0x7fd57006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.161+0000 7fd589400640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd57006be30 0x7fd57006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd5940c3560 tx=0x7fd57c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.165+0000 7fd58aa00640 1 -- 172.21.15.47:0/4113282565 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd59415a730 con 0x7fd5940c8b30 2024-04-24T22:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.361+0000 7fd593200640 1 -- 172.21.15.47:0/4113282565 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd59415e400 con 0x7fd57006be30 2024-04-24T22:40:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.369+0000 7fd58aa00640 1 -- 172.21.15.47:0/4113282565 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251928 (secure 0 0 0) 0x7fd59415e400 con 0x7fd57006be30 2024-04-24T22:40:20.375 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:20.376 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:40:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 -- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd57006be30 msgr2=0x7fd57006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd57006be30 0x7fd57006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd5940c3560 tx=0x7fd57c009040 comp rx=0 tx=0).stop 2024-04-24T22:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 -- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8b30 msgr2=0x7fd5940c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8b30 0x7fd5940c47f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd58400cc10 tx=0x7fd58400a430 comp rx=0 tx=0).stop 2024-04-24T22:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 -- 172.21.15.47:0/4113282565 shutdown_connections 2024-04-24T22:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd57006be30 0x7fd57006e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd59415db70 0x7fd59415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5940c8f10 0x7fd5940cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 --2- 172.21.15.47:0/4113282565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5940c8b30 0x7fd5940c47f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 -- 172.21.15.47:0/4113282565 >> 172.21.15.47:0/4113282565 conn(0x7fd5940c9460 msgr2=0x7fd594158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 -- 172.21.15.47:0/4113282565 shutdown_connections 2024-04-24T22:40:20.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.376+0000 7fd593200640 1 -- 172.21.15.47:0/4113282565 wait complete. 2024-04-24T22:40:20.397 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-24T22:40:18.650398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":364,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235880,"kb_used_data":3112,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738840,"statfs":{"total":773094113280,"available":772852572160,"internally_reserved":0,"allocated":3186688,"data_stored":1801644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002648"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706276+0000","last_change":"2024-04-24T22:39:37.699192+0000","last_active":"2024-04-24T22:39:38.706276+0000","last_peered":"2024-04-24T22:39:38.706276+0000","last_clean":"2024-04-24T22:39:38.706276+0000","last_became_active":"2024-04-24T22:39:37.698920+0000","last_became_peered":"2024-04-24T22:39:37.698920+0000","last_unstale":"2024-04-24T22:39:38.706276+0000","last_undegraded":"2024-04-24T22:39:38.706276+0000","last_fullsized":"2024-04-24T22:39:38.706276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.821236+0000","last_change":"2024-04-24T22:39:37.697245+0000","last_active":"2024-04-24T22:39:38.821236+0000","last_peered":"2024-04-24T22:39:38.821236+0000","last_clean":"2024-04-24T22:39:38.821236+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:39:38.821236+0000","last_undegraded":"2024-04-24T22:39:38.821236+0000","last_fullsized":"2024-04-24T22:39:38.821236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705212+0000","last_change":"2024-04-24T22:39:37.709539+0000","last_active":"2024-04-24T22:39:38.705212+0000","last_peered":"2024-04-24T22:39:38.705212+0000","last_clean":"2024-04-24T22:39:38.705212+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:39:38.705212+0000","last_undegraded":"2024-04-24T22:39:38.705212+0000","last_fullsized":"2024-04-24T22:39:38.705212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820523+0000","last_change":"2024-04-24T22:39:37.719136+0000","last_active":"2024-04-24T22:39:38.820523+0000","last_peered":"2024-04-24T22:39:38.820523+0000","last_clean":"2024-04-24T22:39:38.820523+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:39:38.820523+0000","last_undegraded":"2024-04-24T22:39:38.820523+0000","last_fullsized":"2024-04-24T22:39:38.820523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705940+0000","last_change":"2024-04-24T22:39:37.704671+0000","last_active":"2024-04-24T22:39:38.705940+0000","last_peered":"2024-04-24T22:39:38.705940+0000","last_clean":"2024-04-24T22:39:38.705940+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:39:38.705940+0000","last_undegraded":"2024-04-24T22:39:38.705940+0000","last_fullsized":"2024-04-24T22:39:38.705940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820279+0000","last_change":"2024-04-24T22:39:37.703880+0000","last_active":"2024-04-24T22:39:38.820279+0000","last_peered":"2024-04-24T22:39:38.820279+0000","last_clean":"2024-04-24T22:39:38.820279+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:39:38.820279+0000","last_undegraded":"2024-04-24T22:39:38.820279+0000","last_fullsized":"2024-04-24T22:39:38.820279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705748+0000","last_change":"2024-04-24T22:39:37.703539+0000","last_active":"2024-04-24T22:39:38.705748+0000","last_peered":"2024-04-24T22:39:38.705748+0000","last_clean":"2024-04-24T22:39:38.705748+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:39:38.705748+0000","last_undegraded":"2024-04-24T22:39:38.705748+0000","last_fullsized":"2024-04-24T22:39:38.705748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707950+0000","last_change":"2024-04-24T22:39:37.713543+0000","last_active":"2024-04-24T22:39:38.707950+0000","last_peered":"2024-04-24T22:39:38.707950+0000","last_clean":"2024-04-24T22:39:38.707950+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:39:38.707950+0000","last_undegraded":"2024-04-24T22:39:38.707950+0000","last_fullsized":"2024-04-24T22:39:38.707950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706353+0000","last_change":"2024-04-24T22:39:37.706715+0000","last_active":"2024-04-24T22:39:38.706353+0000","last_peered":"2024-04-24T22:39:38.706353+0000","last_clean":"2024-04-24T22:39:38.706353+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:39:38.706353+0000","last_undegraded":"2024-04-24T22:39:38.706353+0000","last_fullsized":"2024-04-24T22:39:38.706353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819558+0000","last_change":"2024-04-24T22:39:37.701250+0000","last_active":"2024-04-24T22:39:38.819558+0000","last_peered":"2024-04-24T22:39:38.819558+0000","last_clean":"2024-04-24T22:39:38.819558+0000","last_became_active":"2024-04-24T22:39:37.700959+0000","last_became_peered":"2024-04-24T22:39:37.700959+0000","last_unstale":"2024-04-24T22:39:38.819558+0000","last_undegraded":"2024-04-24T22:39:38.819558+0000","last_fullsized":"2024-04-24T22:39:38.819558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819186+0000","last_change":"2024-04-24T22:39:37.701069+0000","last_active":"2024-04-24T22:39:38.819186+0000","last_peered":"2024-04-24T22:39:38.819186+0000","last_clean":"2024-04-24T22:39:38.819186+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:39:38.819186+0000","last_undegraded":"2024-04-24T22:39:38.819186+0000","last_fullsized":"2024-04-24T22:39:38.819186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819923+0000","last_change":"2024-04-24T22:39:37.693508+0000","last_active":"2024-04-24T22:39:38.819923+0000","last_peered":"2024-04-24T22:39:38.819923+0000","last_clean":"2024-04-24T22:39:38.819923+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:39:38.819923+0000","last_undegraded":"2024-04-24T22:39:38.819923+0000","last_fullsized":"2024-04-24T22:39:38.819923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708270+0000","last_change":"2024-04-24T22:39:37.705157+0000","last_active":"2024-04-24T22:39:38.708270+0000","last_peered":"2024-04-24T22:39:38.708270+0000","last_clean":"2024-04-24T22:39:38.708270+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:39:38.708270+0000","last_undegraded":"2024-04-24T22:39:38.708270+0000","last_fullsized":"2024-04-24T22:39:38.708270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704755+0000","last_change":"2024-04-24T22:39:37.698333+0000","last_active":"2024-04-24T22:39:38.704755+0000","last_peered":"2024-04-24T22:39:38.704755+0000","last_clean":"2024-04-24T22:39:38.704755+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:39:38.704755+0000","last_undegraded":"2024-04-24T22:39:38.704755+0000","last_fullsized":"2024-04-24T22:39:38.704755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:57.510197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820219+0000","last_change":"2024-04-24T22:39:37.696874+0000","last_active":"2024-04-24T22:39:38.820219+0000","last_peered":"2024-04-24T22:39:38.820219+0000","last_clean":"2024-04-24T22:39:38.820219+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:39:38.820219+0000","last_undegraded":"2024-04-24T22:39:38.820219+0000","last_fullsized":"2024-04-24T22:39:38.820219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820021+0000","last_change":"2024-04-24T22:39:37.703446+0000","last_active":"2024-04-24T22:39:38.820021+0000","last_peered":"2024-04-24T22:39:38.820021+0000","last_clean":"2024-04-24T22:39:38.820021+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:39:38.820021+0000","last_undegraded":"2024-04-24T22:39:38.820021+0000","last_fullsized":"2024-04-24T22:39:38.820021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820971+0000","last_change":"2024-04-24T22:39:37.701515+0000","last_active":"2024-04-24T22:39:38.820971+0000","last_peered":"2024-04-24T22:39:38.820971+0000","last_clean":"2024-04-24T22:39:38.820971+0000","last_became_active":"2024-04-24T22:39:37.700439+0000","last_became_peered":"2024-04-24T22:39:37.700439+0000","last_unstale":"2024-04-24T22:39:38.820971+0000","last_undegraded":"2024-04-24T22:39:38.820971+0000","last_fullsized":"2024-04-24T22:39:38.820971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819628+0000","last_change":"2024-04-24T22:39:37.703899+0000","last_active":"2024-04-24T22:39:38.819628+0000","last_peered":"2024-04-24T22:39:38.819628+0000","last_clean":"2024-04-24T22:39:38.819628+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:39:38.819628+0000","last_undegraded":"2024-04-24T22:39:38.819628+0000","last_fullsized":"2024-04-24T22:39:38.819628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706244+0000","last_change":"2024-04-24T22:39:37.711019+0000","last_active":"2024-04-24T22:39:38.706244+0000","last_peered":"2024-04-24T22:39:38.706244+0000","last_clean":"2024-04-24T22:39:38.706244+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:39:38.706244+0000","last_undegraded":"2024-04-24T22:39:38.706244+0000","last_fullsized":"2024-04-24T22:39:38.706244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819357+0000","last_change":"2024-04-24T22:39:37.696214+0000","last_active":"2024-04-24T22:39:38.819357+0000","last_peered":"2024-04-24T22:39:38.819357+0000","last_clean":"2024-04-24T22:39:38.819357+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:39:38.819357+0000","last_undegraded":"2024-04-24T22:39:38.819357+0000","last_fullsized":"2024-04-24T22:39:38.819357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705029+0000","last_change":"2024-04-24T22:39:37.700151+0000","last_active":"2024-04-24T22:39:38.705029+0000","last_peered":"2024-04-24T22:39:38.705029+0000","last_clean":"2024-04-24T22:39:38.705029+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:39:38.705029+0000","last_undegraded":"2024-04-24T22:39:38.705029+0000","last_fullsized":"2024-04-24T22:39:38.705029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820267+0000","last_change":"2024-04-24T22:39:37.705083+0000","last_active":"2024-04-24T22:39:38.820267+0000","last_peered":"2024-04-24T22:39:38.820267+0000","last_clean":"2024-04-24T22:39:38.820267+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:39:38.820267+0000","last_undegraded":"2024-04-24T22:39:38.820267+0000","last_fullsized":"2024-04-24T22:39:38.820267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708194+0000","last_change":"2024-04-24T22:39:37.711193+0000","last_active":"2024-04-24T22:39:38.708194+0000","last_peered":"2024-04-24T22:39:38.708194+0000","last_clean":"2024-04-24T22:39:38.708194+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:39:38.708194+0000","last_undegraded":"2024-04-24T22:39:38.708194+0000","last_fullsized":"2024-04-24T22:39:38.708194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820128+0000","last_change":"2024-04-24T22:39:37.713910+0000","last_active":"2024-04-24T22:39:38.820128+0000","last_peered":"2024-04-24T22:39:38.820128+0000","last_clean":"2024-04-24T22:39:38.820128+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:39:38.820128+0000","last_undegraded":"2024-04-24T22:39:38.820128+0000","last_fullsized":"2024-04-24T22:39:38.820128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708266+0000","last_change":"2024-04-24T22:39:37.717191+0000","last_active":"2024-04-24T22:39:38.708266+0000","last_peered":"2024-04-24T22:39:38.708266+0000","last_clean":"2024-04-24T22:39:38.708266+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:39:38.708266+0000","last_undegraded":"2024-04-24T22:39:38.708266+0000","last_fullsized":"2024-04-24T22:39:38.708266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819555+0000","last_change":"2024-04-24T22:39:37.702983+0000","last_active":"2024-04-24T22:39:38.819555+0000","last_peered":"2024-04-24T22:39:38.819555+0000","last_clean":"2024-04-24T22:39:38.819555+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:39:38.819555+0000","last_undegraded":"2024-04-24T22:39:38.819555+0000","last_fullsized":"2024-04-24T22:39:38.819555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820583+0000","last_change":"2024-04-24T22:39:37.714111+0000","last_active":"2024-04-24T22:39:38.820583+0000","last_peered":"2024-04-24T22:39:38.820583+0000","last_clean":"2024-04-24T22:39:38.820583+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:39:38.820583+0000","last_undegraded":"2024-04-24T22:39:38.820583+0000","last_fullsized":"2024-04-24T22:39:38.820583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708923+0000","last_change":"2024-04-24T22:39:37.714877+0000","last_active":"2024-04-24T22:39:38.708923+0000","last_peered":"2024-04-24T22:39:38.708923+0000","last_clean":"2024-04-24T22:39:38.708923+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:39:38.708923+0000","last_undegraded":"2024-04-24T22:39:38.708923+0000","last_fullsized":"2024-04-24T22:39:38.708923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705236+0000","last_change":"2024-04-24T22:39:37.714695+0000","last_active":"2024-04-24T22:39:38.705236+0000","last_peered":"2024-04-24T22:39:38.705236+0000","last_clean":"2024-04-24T22:39:38.705236+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:39:38.705236+0000","last_undegraded":"2024-04-24T22:39:38.705236+0000","last_fullsized":"2024-04-24T22:39:38.705236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820461+0000","last_change":"2024-04-24T22:39:37.719187+0000","last_active":"2024-04-24T22:39:38.820461+0000","last_peered":"2024-04-24T22:39:38.820461+0000","last_clean":"2024-04-24T22:39:38.820461+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:39:38.820461+0000","last_undegraded":"2024-04-24T22:39:38.820461+0000","last_fullsized":"2024-04-24T22:39:38.820461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819723+0000","last_change":"2024-04-24T22:39:37.707283+0000","last_active":"2024-04-24T22:39:38.819723+0000","last_peered":"2024-04-24T22:39:38.819723+0000","last_clean":"2024-04-24T22:39:38.819723+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:39:38.819723+0000","last_undegraded":"2024-04-24T22:39:38.819723+0000","last_fullsized":"2024-04-24T22:39:38.819723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818968+0000","last_change":"2024-04-24T22:39:37.697902+0000","last_active":"2024-04-24T22:39:38.818968+0000","last_peered":"2024-04-24T22:39:38.818968+0000","last_clean":"2024-04-24T22:39:38.818968+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:39:38.818968+0000","last_undegraded":"2024-04-24T22:39:38.818968+0000","last_fullsized":"2024-04-24T22:39:38.818968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819142+0000","last_change":"2024-04-24T22:39:37.705667+0000","last_active":"2024-04-24T22:39:38.819142+0000","last_peered":"2024-04-24T22:39:38.819142+0000","last_clean":"2024-04-24T22:39:38.819142+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:39:38.819142+0000","last_undegraded":"2024-04-24T22:39:38.819142+0000","last_fullsized":"2024-04-24T22:39:38.819142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:50.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:48.752812+0000","last_change":"2024-04-24T22:39:36.707990+0000","last_active":"2024-04-24T22:39:48.752812+0000","last_peered":"2024-04-24T22:39:48.752812+0000","last_clean":"2024-04-24T22:39:48.752812+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:39:48.752812+0000","last_undegraded":"2024-04-24T22:39:48.752812+0000","last_fullsized":"2024-04-24T22:39:48.752812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706420+0000","last_change":"2024-04-24T22:39:37.700406+0000","last_active":"2024-04-24T22:39:38.706420+0000","last_peered":"2024-04-24T22:39:38.706420+0000","last_clean":"2024-04-24T22:39:38.706420+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:39:38.706420+0000","last_undegraded":"2024-04-24T22:39:38.706420+0000","last_fullsized":"2024-04-24T22:39:38.706420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.821287+0000","last_change":"2024-04-24T22:39:36.701398+0000","last_active":"2024-04-24T22:39:59.821287+0000","last_peered":"2024-04-24T22:39:59.821287+0000","last_clean":"2024-04-24T22:39:59.821287+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:39:59.821287+0000","last_undegraded":"2024-04-24T22:39:59.821287+0000","last_fullsized":"2024-04-24T22:39:59.821287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:53.780440+0000","last_change":"2024-04-24T22:39:36.712294+0000","last_active":"2024-04-24T22:39:53.780440+0000","last_peered":"2024-04-24T22:39:53.780440+0000","last_clean":"2024-04-24T22:39:53.780440+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:39:53.780440+0000","last_undegraded":"2024-04-24T22:39:53.780440+0000","last_fullsized":"2024-04-24T22:39:53.780440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707809+0000","last_change":"2024-04-24T22:39:37.716920+0000","last_active":"2024-04-24T22:39:38.707809+0000","last_peered":"2024-04-24T22:39:38.707809+0000","last_clean":"2024-04-24T22:39:38.707809+0000","last_became_active":"2024-04-24T22:39:37.716641+0000","last_became_peered":"2024-04-24T22:39:37.716641+0000","last_unstale":"2024-04-24T22:39:38.707809+0000","last_undegraded":"2024-04-24T22:39:38.707809+0000","last_fullsized":"2024-04-24T22:39:38.707809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.816798+0000","last_change":"2024-04-24T22:39:36.698983+0000","last_active":"2024-04-24T22:39:59.816798+0000","last_peered":"2024-04-24T22:39:59.816798+0000","last_clean":"2024-04-24T22:39:59.816798+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:39:59.816798+0000","last_undegraded":"2024-04-24T22:39:59.816798+0000","last_fullsized":"2024-04-24T22:39:59.816798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820135+0000","last_change":"2024-04-24T22:39:37.714229+0000","last_active":"2024-04-24T22:39:38.820135+0000","last_peered":"2024-04-24T22:39:38.820135+0000","last_clean":"2024-04-24T22:39:38.820135+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:39:38.820135+0000","last_undegraded":"2024-04-24T22:39:38.820135+0000","last_fullsized":"2024-04-24T22:39:38.820135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:48.752647+0000","last_change":"2024-04-24T22:39:36.705776+0000","last_active":"2024-04-24T22:39:48.752647+0000","last_peered":"2024-04-24T22:39:48.752647+0000","last_clean":"2024-04-24T22:39:48.752647+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:39:48.752647+0000","last_undegraded":"2024-04-24T22:39:48.752647+0000","last_fullsized":"2024-04-24T22:39:48.752647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820936+0000","last_change":"2024-04-24T22:39:37.701436+0000","last_active":"2024-04-24T22:39:38.820936+0000","last_peered":"2024-04-24T22:39:38.820936+0000","last_clean":"2024-04-24T22:39:38.820936+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:39:38.820936+0000","last_undegraded":"2024-04-24T22:39:38.820936+0000","last_fullsized":"2024-04-24T22:39:38.820936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.843014+0000","last_change":"2024-04-24T22:39:36.700237+0000","last_active":"2024-04-24T22:40:05.843014+0000","last_peered":"2024-04-24T22:40:05.843014+0000","last_clean":"2024-04-24T22:40:05.843014+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:40:05.843014+0000","last_undegraded":"2024-04-24T22:40:05.843014+0000","last_fullsized":"2024-04-24T22:40:05.843014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705984+0000","last_change":"2024-04-24T22:39:37.703532+0000","last_active":"2024-04-24T22:39:38.705984+0000","last_peered":"2024-04-24T22:39:38.705984+0000","last_clean":"2024-04-24T22:39:38.705984+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:39:38.705984+0000","last_undegraded":"2024-04-24T22:39:38.705984+0000","last_fullsized":"2024-04-24T22:39:38.705984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:48.750425+0000","last_change":"2024-04-24T22:39:36.717477+0000","last_active":"2024-04-24T22:39:48.750425+0000","last_peered":"2024-04-24T22:39:48.750425+0000","last_clean":"2024-04-24T22:39:48.750425+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:39:48.750425+0000","last_undegraded":"2024-04-24T22:39:48.750425+0000","last_fullsized":"2024-04-24T22:39:48.750425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819989+0000","last_change":"2024-04-24T22:39:37.699255+0000","last_active":"2024-04-24T22:39:38.819989+0000","last_peered":"2024-04-24T22:39:38.819989+0000","last_clean":"2024-04-24T22:39:38.819989+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:39:38.819989+0000","last_undegraded":"2024-04-24T22:39:38.819989+0000","last_fullsized":"2024-04-24T22:39:38.819989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'10","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.842727+0000","last_change":"2024-04-24T22:39:36.706567+0000","last_active":"2024-04-24T22:40:05.842727+0000","last_peered":"2024-04-24T22:40:05.842727+0000","last_clean":"2024-04-24T22:40:05.842727+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:40:05.842727+0000","last_undegraded":"2024-04-24T22:40:05.842727+0000","last_fullsized":"2024-04-24T22:40:05.842727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706118+0000","last_change":"2024-04-24T22:39:37.712064+0000","last_active":"2024-04-24T22:39:38.706118+0000","last_peered":"2024-04-24T22:39:38.706118+0000","last_clean":"2024-04-24T22:39:38.706118+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:39:38.706118+0000","last_undegraded":"2024-04-24T22:39:38.706118+0000","last_fullsized":"2024-04-24T22:39:38.706118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'9","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.842540+0000","last_change":"2024-04-24T22:39:36.700041+0000","last_active":"2024-04-24T22:40:05.842540+0000","last_peered":"2024-04-24T22:40:05.842540+0000","last_clean":"2024-04-24T22:40:05.842540+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:40:05.842540+0000","last_undegraded":"2024-04-24T22:40:05.842540+0000","last_fullsized":"2024-04-24T22:40:05.842540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:50.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705269+0000","last_change":"2024-04-24T22:39:37.691459+0000","last_active":"2024-04-24T22:39:38.705269+0000","last_peered":"2024-04-24T22:39:38.705269+0000","last_clean":"2024-04-24T22:39:38.705269+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:39:38.705269+0000","last_undegraded":"2024-04-24T22:39:38.705269+0000","last_fullsized":"2024-04-24T22:39:38.705269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.822296+0000","last_change":"2024-04-24T22:39:36.706454+0000","last_active":"2024-04-24T22:39:59.822296+0000","last_peered":"2024-04-24T22:39:59.822296+0000","last_clean":"2024-04-24T22:39:59.822296+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:39:59.822296+0000","last_undegraded":"2024-04-24T22:39:59.822296+0000","last_fullsized":"2024-04-24T22:39:59.822296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705898+0000","last_change":"2024-04-24T22:39:37.696724+0000","last_active":"2024-04-24T22:39:38.705898+0000","last_peered":"2024-04-24T22:39:38.705898+0000","last_clean":"2024-04-24T22:39:38.705898+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:39:38.705898+0000","last_undegraded":"2024-04-24T22:39:38.705898+0000","last_fullsized":"2024-04-24T22:39:38.705898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.820803+0000","last_change":"2024-04-24T22:39:36.701340+0000","last_active":"2024-04-24T22:39:59.820803+0000","last_peered":"2024-04-24T22:39:59.820803+0000","last_clean":"2024-04-24T22:39:59.820803+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:39:59.820803+0000","last_undegraded":"2024-04-24T22:39:59.820803+0000","last_fullsized":"2024-04-24T22:39:59.820803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819009+0000","last_change":"2024-04-24T22:39:37.697532+0000","last_active":"2024-04-24T22:39:38.819009+0000","last_peered":"2024-04-24T22:39:38.819009+0000","last_clean":"2024-04-24T22:39:38.819009+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:39:38.819009+0000","last_undegraded":"2024-04-24T22:39:38.819009+0000","last_fullsized":"2024-04-24T22:39:38.819009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'17","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.825168+0000","last_change":"2024-04-24T22:39:36.724649+0000","last_active":"2024-04-24T22:39:59.825168+0000","last_peered":"2024-04-24T22:39:59.825168+0000","last_clean":"2024-04-24T22:39:59.825168+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:39:59.825168+0000","last_undegraded":"2024-04-24T22:39:59.825168+0000","last_fullsized":"2024-04-24T22:39:59.825168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705839+0000","last_change":"2024-04-24T22:39:37.702890+0000","last_active":"2024-04-24T22:39:38.705839+0000","last_peered":"2024-04-24T22:39:38.705839+0000","last_clean":"2024-04-24T22:39:38.705839+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:39:38.705839+0000","last_undegraded":"2024-04-24T22:39:38.705839+0000","last_fullsized":"2024-04-24T22:39:38.705839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.720176+0000","last_change":"2024-04-24T22:39:36.716044+0000","last_active":"2024-04-24T22:39:42.720176+0000","last_peered":"2024-04-24T22:39:42.720176+0000","last_clean":"2024-04-24T22:39:42.720176+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:39:42.720176+0000","last_undegraded":"2024-04-24T22:39:42.720176+0000","last_fullsized":"2024-04-24T22:39:42.720176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:40.023933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706077+0000","last_change":"2024-04-24T22:39:37.702621+0000","last_active":"2024-04-24T22:39:38.706077+0000","last_peered":"2024-04-24T22:39:38.706077+0000","last_clean":"2024-04-24T22:39:38.706077+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:39:38.706077+0000","last_undegraded":"2024-04-24T22:39:38.706077+0000","last_fullsized":"2024-04-24T22:39:38.706077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.843760+0000","last_change":"2024-04-24T22:39:36.714272+0000","last_active":"2024-04-24T22:40:05.843760+0000","last_peered":"2024-04-24T22:40:05.843760+0000","last_clean":"2024-04-24T22:40:05.843760+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:40:05.843760+0000","last_undegraded":"2024-04-24T22:40:05.843760+0000","last_fullsized":"2024-04-24T22:40:05.843760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706373+0000","last_change":"2024-04-24T22:39:37.703786+0000","last_active":"2024-04-24T22:39:38.706373+0000","last_peered":"2024-04-24T22:39:38.706373+0000","last_clean":"2024-04-24T22:39:38.706373+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:39:38.706373+0000","last_undegraded":"2024-04-24T22:39:38.706373+0000","last_fullsized":"2024-04-24T22:39:38.706373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:55.990656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.823637+0000","last_change":"2024-04-24T22:39:36.715437+0000","last_active":"2024-04-24T22:39:59.823637+0000","last_peered":"2024-04-24T22:39:59.823637+0000","last_clean":"2024-04-24T22:39:59.823637+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:39:59.823637+0000","last_undegraded":"2024-04-24T22:39:59.823637+0000","last_fullsized":"2024-04-24T22:39:59.823637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820735+0000","last_change":"2024-04-24T22:39:37.715801+0000","last_active":"2024-04-24T22:39:38.820735+0000","last_peered":"2024-04-24T22:39:38.820735+0000","last_clean":"2024-04-24T22:39:38.820735+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:39:38.820735+0000","last_undegraded":"2024-04-24T22:39:38.820735+0000","last_fullsized":"2024-04-24T22:39:38.820735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.822281+0000","last_change":"2024-04-24T22:39:36.710422+0000","last_active":"2024-04-24T22:39:59.822281+0000","last_peered":"2024-04-24T22:39:59.822281+0000","last_clean":"2024-04-24T22:39:59.822281+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:39:59.822281+0000","last_undegraded":"2024-04-24T22:39:59.822281+0000","last_fullsized":"2024-04-24T22:39:59.822281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706031+0000","last_change":"2024-04-24T22:39:37.706745+0000","last_active":"2024-04-24T22:39:38.706031+0000","last_peered":"2024-04-24T22:39:38.706031+0000","last_clean":"2024-04-24T22:39:38.706031+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:39:38.706031+0000","last_undegraded":"2024-04-24T22:39:38.706031+0000","last_fullsized":"2024-04-24T22:39:38.706031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705671+0000","last_change":"2024-04-24T22:39:32.657965+0000","last_active":"2024-04-24T22:39:38.705671+0000","last_peered":"2024-04-24T22:39:38.705671+0000","last_clean":"2024-04-24T22:39:38.705671+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:39:38.705671+0000","last_undegraded":"2024-04-24T22:39:38.705671+0000","last_fullsized":"2024-04-24T22:39:38.705671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_clean_scrub_stamp":"2024-04-24T22:39:31.635804+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.714153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.822693+0000","last_change":"2024-04-24T22:39:36.710381+0000","last_active":"2024-04-24T22:39:59.822693+0000","last_peered":"2024-04-24T22:39:59.822693+0000","last_clean":"2024-04-24T22:39:59.822693+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:39:59.822693+0000","last_undegraded":"2024-04-24T22:39:59.822693+0000","last_fullsized":"2024-04-24T22:39:59.822693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708707+0000","last_change":"2024-04-24T22:39:37.713956+0000","last_active":"2024-04-24T22:39:38.708707+0000","last_peered":"2024-04-24T22:39:38.708707+0000","last_clean":"2024-04-24T22:39:38.708707+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:39:38.708707+0000","last_undegraded":"2024-04-24T22:39:38.708707+0000","last_fullsized":"2024-04-24T22:39:38.708707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:53.782609+0000","last_change":"2024-04-24T22:39:36.715290+0000","last_active":"2024-04-24T22:39:53.782609+0000","last_peered":"2024-04-24T22:39:53.782609+0000","last_clean":"2024-04-24T22:39:53.782609+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:39:53.782609+0000","last_undegraded":"2024-04-24T22:39:53.782609+0000","last_fullsized":"2024-04-24T22:39:53.782609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704538+0000","last_change":"2024-04-24T22:39:37.695419+0000","last_active":"2024-04-24T22:39:38.704538+0000","last_peered":"2024-04-24T22:39:38.704538+0000","last_clean":"2024-04-24T22:39:38.704538+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:39:38.704538+0000","last_undegraded":"2024-04-24T22:39:38.704538+0000","last_fullsized":"2024-04-24T22:39:38.704538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:08.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:53.785443+0000","last_change":"2024-04-24T22:39:36.707548+0000","last_active":"2024-04-24T22:39:53.785443+0000","last_peered":"2024-04-24T22:39:53.785443+0000","last_clean":"2024-04-24T22:39:53.785443+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:39:53.785443+0000","last_undegraded":"2024-04-24T22:39:53.785443+0000","last_fullsized":"2024-04-24T22:39:53.785443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820375+0000","last_change":"2024-04-24T22:39:37.703301+0000","last_active":"2024-04-24T22:39:38.820375+0000","last_peered":"2024-04-24T22:39:38.820375+0000","last_clean":"2024-04-24T22:39:38.820375+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:39:38.820375+0000","last_undegraded":"2024-04-24T22:39:38.820375+0000","last_fullsized":"2024-04-24T22:39:38.820375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.844950+0000","last_change":"2024-04-24T22:39:36.713486+0000","last_active":"2024-04-24T22:40:05.844950+0000","last_peered":"2024-04-24T22:40:05.844950+0000","last_clean":"2024-04-24T22:40:05.844950+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:40:05.844950+0000","last_undegraded":"2024-04-24T22:40:05.844950+0000","last_fullsized":"2024-04-24T22:40:05.844950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819526+0000","last_change":"2024-04-24T22:39:37.705705+0000","last_active":"2024-04-24T22:39:38.819526+0000","last_peered":"2024-04-24T22:39:38.819526+0000","last_clean":"2024-04-24T22:39:38.819526+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:39:38.819526+0000","last_undegraded":"2024-04-24T22:39:38.819526+0000","last_fullsized":"2024-04-24T22:39:38.819526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:34.424877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.686949+0000","last_change":"2024-04-24T22:39:36.723125+0000","last_active":"2024-04-24T22:39:39.686949+0000","last_peered":"2024-04-24T22:39:39.686949+0000","last_clean":"2024-04-24T22:39:39.686949+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:39:39.686949+0000","last_undegraded":"2024-04-24T22:39:39.686949+0000","last_fullsized":"2024-04-24T22:39:39.686949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820922+0000","last_change":"2024-04-24T22:39:37.713947+0000","last_active":"2024-04-24T22:39:38.820922+0000","last_peered":"2024-04-24T22:39:38.820922+0000","last_clean":"2024-04-24T22:39:38.820922+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:39:38.820922+0000","last_undegraded":"2024-04-24T22:39:38.820922+0000","last_fullsized":"2024-04-24T22:39:38.820922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.821515+0000","last_change":"2024-04-24T22:39:36.716541+0000","last_active":"2024-04-24T22:39:59.821515+0000","last_peered":"2024-04-24T22:39:59.821515+0000","last_clean":"2024-04-24T22:39:59.821515+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:39:59.821515+0000","last_undegraded":"2024-04-24T22:39:59.821515+0000","last_fullsized":"2024-04-24T22:39:59.821515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:33.366773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820580+0000","last_change":"2024-04-24T22:39:37.708069+0000","last_active":"2024-04-24T22:39:38.820580+0000","last_peered":"2024-04-24T22:39:38.820580+0000","last_clean":"2024-04-24T22:39:38.820580+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:39:38.820580+0000","last_undegraded":"2024-04-24T22:39:38.820580+0000","last_fullsized":"2024-04-24T22:39:38.820580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.824116+0000","last_change":"2024-04-24T22:39:36.718740+0000","last_active":"2024-04-24T22:39:59.824116+0000","last_peered":"2024-04-24T22:39:59.824116+0000","last_clean":"2024-04-24T22:39:59.824116+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:39:59.824116+0000","last_undegraded":"2024-04-24T22:39:59.824116+0000","last_fullsized":"2024-04-24T22:39:59.824116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.709326+0000","last_change":"2024-04-24T22:39:37.704496+0000","last_active":"2024-04-24T22:39:38.709326+0000","last_peered":"2024-04-24T22:39:38.709326+0000","last_clean":"2024-04-24T22:39:38.709326+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:39:38.709326+0000","last_undegraded":"2024-04-24T22:39:38.709326+0000","last_fullsized":"2024-04-24T22:39:38.709326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.841962+0000","last_change":"2024-04-24T22:39:36.701308+0000","last_active":"2024-04-24T22:40:05.841962+0000","last_peered":"2024-04-24T22:40:05.841962+0000","last_clean":"2024-04-24T22:40:05.841962+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:40:05.841962+0000","last_undegraded":"2024-04-24T22:40:05.841962+0000","last_fullsized":"2024-04-24T22:40:05.841962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705483+0000","last_change":"2024-04-24T22:39:37.700189+0000","last_active":"2024-04-24T22:39:38.705483+0000","last_peered":"2024-04-24T22:39:38.705483+0000","last_clean":"2024-04-24T22:39:38.705483+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:39:38.705483+0000","last_undegraded":"2024-04-24T22:39:38.705483+0000","last_fullsized":"2024-04-24T22:39:38.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:48.749442+0000","last_change":"2024-04-24T22:39:36.716460+0000","last_active":"2024-04-24T22:39:48.749442+0000","last_peered":"2024-04-24T22:39:48.749442+0000","last_clean":"2024-04-24T22:39:48.749442+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:39:48.749442+0000","last_undegraded":"2024-04-24T22:39:48.749442+0000","last_fullsized":"2024-04-24T22:39:48.749442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819020+0000","last_change":"2024-04-24T22:39:37.704143+0000","last_active":"2024-04-24T22:39:38.819020+0000","last_peered":"2024-04-24T22:39:38.819020+0000","last_clean":"2024-04-24T22:39:38.819020+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:39:38.819020+0000","last_undegraded":"2024-04-24T22:39:38.819020+0000","last_fullsized":"2024-04-24T22:39:38.819020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:58.556519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.844145+0000","last_change":"2024-04-24T22:39:36.721540+0000","last_active":"2024-04-24T22:40:05.844145+0000","last_peered":"2024-04-24T22:40:05.844145+0000","last_clean":"2024-04-24T22:40:05.844145+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:40:05.844145+0000","last_undegraded":"2024-04-24T22:40:05.844145+0000","last_fullsized":"2024-04-24T22:40:05.844145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708008+0000","last_change":"2024-04-24T22:39:37.714150+0000","last_active":"2024-04-24T22:39:38.708008+0000","last_peered":"2024-04-24T22:39:38.708008+0000","last_clean":"2024-04-24T22:39:38.708008+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:39:38.708008+0000","last_undegraded":"2024-04-24T22:39:38.708008+0000","last_fullsized":"2024-04-24T22:39:38.708008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.823570+0000","last_change":"2024-04-24T22:39:36.716667+0000","last_active":"2024-04-24T22:39:59.823570+0000","last_peered":"2024-04-24T22:39:59.823570+0000","last_clean":"2024-04-24T22:39:59.823570+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:39:59.823570+0000","last_undegraded":"2024-04-24T22:39:59.823570+0000","last_fullsized":"2024-04-24T22:39:59.823570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708586+0000","last_change":"2024-04-24T22:39:37.715906+0000","last_active":"2024-04-24T22:39:38.708586+0000","last_peered":"2024-04-24T22:39:38.708586+0000","last_clean":"2024-04-24T22:39:38.708586+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:39:38.708586+0000","last_undegraded":"2024-04-24T22:39:38.708586+0000","last_fullsized":"2024-04-24T22:39:38.708586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:48.748623+0000","last_change":"2024-04-24T22:39:36.698212+0000","last_active":"2024-04-24T22:39:48.748623+0000","last_peered":"2024-04-24T22:39:48.748623+0000","last_clean":"2024-04-24T22:39:48.748623+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:39:48.748623+0000","last_undegraded":"2024-04-24T22:39:48.748623+0000","last_fullsized":"2024-04-24T22:39:48.748623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818963+0000","last_change":"2024-04-24T22:39:37.696143+0000","last_active":"2024-04-24T22:39:38.818963+0000","last_peered":"2024-04-24T22:39:38.818963+0000","last_clean":"2024-04-24T22:39:38.818963+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:39:38.818963+0000","last_undegraded":"2024-04-24T22:39:38.818963+0000","last_fullsized":"2024-04-24T22:39:38.818963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820834+0000","last_change":"2024-04-24T22:39:37.696113+0000","last_active":"2024-04-24T22:39:38.820834+0000","last_peered":"2024-04-24T22:39:38.820834+0000","last_clean":"2024-04-24T22:39:38.820834+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:39:38.820834+0000","last_undegraded":"2024-04-24T22:39:38.820834+0000","last_fullsized":"2024-04-24T22:39:38.820834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.842425+0000","last_change":"2024-04-24T22:39:36.699149+0000","last_active":"2024-04-24T22:40:05.842425+0000","last_peered":"2024-04-24T22:40:05.842425+0000","last_clean":"2024-04-24T22:40:05.842425+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:40:05.842425+0000","last_undegraded":"2024-04-24T22:40:05.842425+0000","last_fullsized":"2024-04-24T22:40:05.842425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820550+0000","last_change":"2024-04-24T22:39:37.700972+0000","last_active":"2024-04-24T22:39:38.820550+0000","last_peered":"2024-04-24T22:39:38.820550+0000","last_clean":"2024-04-24T22:39:38.820550+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:39:38.820550+0000","last_undegraded":"2024-04-24T22:39:38.820550+0000","last_fullsized":"2024-04-24T22:39:38.820550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'12","reported_seq":26,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.843035+0000","last_change":"2024-04-24T22:39:36.698567+0000","last_active":"2024-04-24T22:40:05.843035+0000","last_peered":"2024-04-24T22:40:05.843035+0000","last_clean":"2024-04-24T22:40:05.843035+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:40:05.843035+0000","last_undegraded":"2024-04-24T22:40:05.843035+0000","last_fullsized":"2024-04-24T22:40:05.843035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706000+0000","last_change":"2024-04-24T22:39:37.706951+0000","last_active":"2024-04-24T22:39:38.706000+0000","last_peered":"2024-04-24T22:39:38.706000+0000","last_clean":"2024-04-24T22:39:38.706000+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:39:38.706000+0000","last_undegraded":"2024-04-24T22:39:38.706000+0000","last_fullsized":"2024-04-24T22:39:38.706000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.729606+0000","last_change":"2024-04-24T22:39:36.703823+0000","last_active":"2024-04-24T22:39:42.729606+0000","last_peered":"2024-04-24T22:39:42.729606+0000","last_clean":"2024-04-24T22:39:42.729606+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:39:42.729606+0000","last_undegraded":"2024-04-24T22:39:42.729606+0000","last_fullsized":"2024-04-24T22:39:42.729606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708601+0000","last_change":"2024-04-24T22:39:37.712078+0000","last_active":"2024-04-24T22:39:38.708601+0000","last_peered":"2024-04-24T22:39:38.708601+0000","last_clean":"2024-04-24T22:39:38.708601+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:39:38.708601+0000","last_undegraded":"2024-04-24T22:39:38.708601+0000","last_fullsized":"2024-04-24T22:39:38.708601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'13","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.843115+0000","last_change":"2024-04-24T22:39:36.719643+0000","last_active":"2024-04-24T22:40:05.843115+0000","last_peered":"2024-04-24T22:40:05.843115+0000","last_clean":"2024-04-24T22:40:05.843115+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:40:05.843115+0000","last_undegraded":"2024-04-24T22:40:05.843115+0000","last_fullsized":"2024-04-24T22:40:05.843115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":205,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":847872,"data_stored":668073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":78007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":79537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":76935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":249856,"data_stored":78041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":249856,"data_stored":77067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":245760,"data_stored":77137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":839680,"data_stored":666847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:40:20.402 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:40:20.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.649+0000 7fd285a00640 1 Processor -- start 2024-04-24T22:40:20.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.649+0000 7fd285a00640 1 -- start start 2024-04-24T22:40:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.651+0000 7fd285a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2800c7750 0x7fd2800c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.651+0000 7fd285a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd280072b80 0x7fd280072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.651+0000 7fd285a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2800c7bb0 con 0x7fd280152960 2024-04-24T22:40:20.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.651+0000 7fd285a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd280158f20 con 0x7fd280072b80 2024-04-24T22:40:20.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.651+0000 7fd285a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd2801593b0 con 0x7fd2800c7750 2024-04-24T22:40:20.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.651+0000 7fd27e000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2800c7750 0x7fd2800c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.651+0000 7fd27ea00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd280152960 0x7fd28015ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56508/0 (socket says 172.21.15.47:56508) 2024-04-24T22:40:20.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.651+0000 7fd27ea00640 1 -- 172.21.15.47:0/71055292 learned_addr learned my addr 172.21.15.47:0/71055292 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:20.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.652+0000 7fd27f400640 1 --2- 172.21.15.47:0/71055292 >> v2:172.21.15.47:3300/0 conn(0x7fd280072b80 0x7fd280072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.652+0000 7fd27e000640 1 -- 172.21.15.47:0/71055292 >> v2:172.21.15.47:3300/0 conn(0x7fd280072b80 msgr2=0x7fd280072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:20.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.652+0000 7fd27e000640 1 --2- 172.21.15.47:0/71055292 >> v2:172.21.15.47:3300/0 conn(0x7fd280072b80 0x7fd280072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.652+0000 7fd27e000640 1 -- 172.21.15.47:0/71055292 >> v1:172.21.15.47:6789/0 conn(0x7fd280152960 legacy=0x7fd28015ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.652+0000 7fd27e000640 1 -- 172.21.15.47:0/71055292 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2800c7e50 con 0x7fd2800c7750 2024-04-24T22:40:20.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.653+0000 7fd27e000640 1 --2- 172.21.15.47:0/71055292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2800c7750 0x7fd2800c8fa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd280156080 tx=0x7fd26c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14da69814b201afa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.653+0000 7fd27d600640 1 -- 172.21.15.47:0/71055292 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c037070 con 0x7fd2800c7750 2024-04-24T22:40:20.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.653+0000 7fd27d600640 1 -- 172.21.15.47:0/71055292 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd26c032440 con 0x7fd2800c7750 2024-04-24T22:40:20.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.653+0000 7fd27d600640 1 -- 172.21.15.47:0/71055292 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c032810 con 0x7fd2800c7750 2024-04-24T22:40:20.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.654+0000 7fd285a00640 1 -- 172.21.15.47:0/71055292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2800c7750 msgr2=0x7fd2800c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.654+0000 7fd285a00640 1 --2- 172.21.15.47:0/71055292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2800c7750 0x7fd2800c8fa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd280156080 tx=0x7fd26c02a250 comp rx=0 tx=0).stop 2024-04-24T22:40:20.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.654+0000 7fd285a00640 1 -- 172.21.15.47:0/71055292 shutdown_connections 2024-04-24T22:40:20.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.654+0000 7fd285a00640 1 --2- 172.21.15.47:0/71055292 >> v2:172.21.15.47:3300/0 conn(0x7fd280072b80 0x7fd280072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.654+0000 7fd285a00640 1 --2- 172.21.15.47:0/71055292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2800c7750 0x7fd2800c8fa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.654+0000 7fd285a00640 1 -- 172.21.15.47:0/71055292 >> 172.21.15.47:0/71055292 conn(0x7fd28006e580 msgr2=0x7fd28006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:20.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.654+0000 7fd285a00640 1 -- 172.21.15.47:0/71055292 shutdown_connections 2024-04-24T22:40:20.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.654+0000 7fd285a00640 1 -- 172.21.15.47:0/71055292 wait complete. 2024-04-24T22:40:20.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.657+0000 7fd285a00640 1 Processor -- start 2024-04-24T22:40:20.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.657+0000 7fd285a00640 1 -- start start 2024-04-24T22:40:20.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.658+0000 7fd285a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd280072b80 0x7fd28014d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.658+0000 7fd285a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800c7750 0x7fd2801436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.658+0000 7fd285a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd280152960 0x7fd2800c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.658+0000 7fd285a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801588d0 con 0x7fd2800c7750 2024-04-24T22:40:20.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.658+0000 7fd285a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd2800c8130 con 0x7fd280152960 2024-04-24T22:40:20.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.658+0000 7fd285a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd280158bb0 con 0x7fd280072b80 2024-04-24T22:40:20.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.658+0000 7fd27ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd280072b80 0x7fd28014d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.659+0000 7fd27ea00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd280072b80 0x7fd28014d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34362/0 (socket says 172.21.15.47:34362) 2024-04-24T22:40:20.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.659+0000 7fd27ea00640 1 -- 172.21.15.47:0/3642950296 learned_addr learned my addr 172.21.15.47:0/3642950296 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:20.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.659+0000 7fd27e000640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800c7750 0x7fd2801436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.659+0000 7fd27f400640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd280152960 0x7fd2800c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.659+0000 7fd27ea00640 1 -- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd280152960 msgr2=0x7fd2800c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.659+0000 7fd27ea00640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd280152960 0x7fd2800c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.659+0000 7fd27ea00640 1 -- 172.21.15.47:0/3642950296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800c7750 msgr2=0x7fd2801436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.659+0000 7fd27ea00640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800c7750 0x7fd2801436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.659+0000 7fd27ea00640 1 -- 172.21.15.47:0/3642950296 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2800c50d0 con 0x7fd280072b80 2024-04-24T22:40:20.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.659+0000 7fd27ea00640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd280072b80 0x7fd28014d270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd27000c3c0 tx=0x7fd27000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.660+0000 7fd276a00640 1 -- 172.21.15.47:0/3642950296 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270017070 con 0x7fd280072b80 2024-04-24T22:40:20.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.660+0000 7fd276a00640 1 -- 172.21.15.47:0/3642950296 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd270005ce0 con 0x7fd280072b80 2024-04-24T22:40:20.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.660+0000 7fd285a00640 1 -- 172.21.15.47:0/3642950296 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2801f3b80 con 0x7fd280072b80 2024-04-24T22:40:20.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.660+0000 7fd276a00640 1 -- 172.21.15.47:0/3642950296 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270007310 con 0x7fd280072b80 2024-04-24T22:40:20.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.660+0000 7fd285a00640 1 -- 172.21.15.47:0/3642950296 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2801f4140 con 0x7fd280072b80 2024-04-24T22:40:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.661+0000 7fd285a00640 1 -- 172.21.15.47:0/3642950296 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd280073b90 con 0x7fd280072b80 2024-04-24T22:40:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.662+0000 7fd276a00640 1 -- 172.21.15.47:0/3642950296 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd27001a070 con 0x7fd280072b80 2024-04-24T22:40:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.663+0000 7fd276a00640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd26006be30 0x7fd26006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.663+0000 7fd276a00640 1 -- 172.21.15.47:0/3642950296 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd27008a1c0 con 0x7fd280072b80 2024-04-24T22:40:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.663+0000 7fd27e000640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd26006be30 0x7fd26006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.664+0000 7fd27e000640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd26006be30 0x7fd26006e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd26c034000 tx=0x7fd26c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.667+0000 7fd276a00640 1 -- 172.21.15.47:0/3642950296 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd2800c50d0 con 0x7fd280072b80 2024-04-24T22:40:20.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.872+0000 7fd285a00640 1 -- 172.21.15.47:0/3642950296 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd28006c200 con 0x7fd26006be30 2024-04-24T22:40:20.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.879+0000 7fd276a00640 1 -- 172.21.15.47:0/3642950296 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251976 (secure 0 0 0) 0x7fd28006c200 con 0x7fd26006be30 2024-04-24T22:40:20.884 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:20.885 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:40:20.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 -- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd26006be30 msgr2=0x7fd26006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd26006be30 0x7fd26006e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd26c034000 tx=0x7fd26c02e000 comp rx=0 tx=0).stop 2024-04-24T22:40:20.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 -- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd280072b80 msgr2=0x7fd28014d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:20.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd280072b80 0x7fd28014d270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd27000c3c0 tx=0x7fd27000c880 comp rx=0 tx=0).stop 2024-04-24T22:40:20.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 -- 172.21.15.47:0/3642950296 shutdown_connections 2024-04-24T22:40:20.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd26006be30 0x7fd26006e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd280152960 0x7fd2800c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2800c7750 0x7fd2801436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 --2- 172.21.15.47:0/3642950296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd280072b80 0x7fd28014d270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:20.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 -- 172.21.15.47:0/3642950296 >> 172.21.15.47:0/3642950296 conn(0x7fd28006e580 msgr2=0x7fd2801519b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:20.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 -- 172.21.15.47:0/3642950296 shutdown_connections 2024-04-24T22:40:20.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:20.885+0000 7fd285a00640 1 -- 172.21.15.47:0/3642950296 wait complete. 2024-04-24T22:40:20.904 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-24T22:40:20.650783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599733,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":368,"num_write_kb":2482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235880,"kb_used_data":3112,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738840,"statfs":{"total":773094113280,"available":772852572160,"internally_reserved":0,"allocated":3186688,"data_stored":1804306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2737,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002546"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706276+0000","last_change":"2024-04-24T22:39:37.699192+0000","last_active":"2024-04-24T22:39:38.706276+0000","last_peered":"2024-04-24T22:39:38.706276+0000","last_clean":"2024-04-24T22:39:38.706276+0000","last_became_active":"2024-04-24T22:39:37.698920+0000","last_became_peered":"2024-04-24T22:39:37.698920+0000","last_unstale":"2024-04-24T22:39:38.706276+0000","last_undegraded":"2024-04-24T22:39:38.706276+0000","last_fullsized":"2024-04-24T22:39:38.706276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.821236+0000","last_change":"2024-04-24T22:39:37.697245+0000","last_active":"2024-04-24T22:39:38.821236+0000","last_peered":"2024-04-24T22:39:38.821236+0000","last_clean":"2024-04-24T22:39:38.821236+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:39:38.821236+0000","last_undegraded":"2024-04-24T22:39:38.821236+0000","last_fullsized":"2024-04-24T22:39:38.821236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705212+0000","last_change":"2024-04-24T22:39:37.709539+0000","last_active":"2024-04-24T22:39:38.705212+0000","last_peered":"2024-04-24T22:39:38.705212+0000","last_clean":"2024-04-24T22:39:38.705212+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:39:38.705212+0000","last_undegraded":"2024-04-24T22:39:38.705212+0000","last_fullsized":"2024-04-24T22:39:38.705212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820523+0000","last_change":"2024-04-24T22:39:37.719136+0000","last_active":"2024-04-24T22:39:38.820523+0000","last_peered":"2024-04-24T22:39:38.820523+0000","last_clean":"2024-04-24T22:39:38.820523+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:39:38.820523+0000","last_undegraded":"2024-04-24T22:39:38.820523+0000","last_fullsized":"2024-04-24T22:39:38.820523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705940+0000","last_change":"2024-04-24T22:39:37.704671+0000","last_active":"2024-04-24T22:39:38.705940+0000","last_peered":"2024-04-24T22:39:38.705940+0000","last_clean":"2024-04-24T22:39:38.705940+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:39:38.705940+0000","last_undegraded":"2024-04-24T22:39:38.705940+0000","last_fullsized":"2024-04-24T22:39:38.705940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820279+0000","last_change":"2024-04-24T22:39:37.703880+0000","last_active":"2024-04-24T22:39:38.820279+0000","last_peered":"2024-04-24T22:39:38.820279+0000","last_clean":"2024-04-24T22:39:38.820279+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:39:38.820279+0000","last_undegraded":"2024-04-24T22:39:38.820279+0000","last_fullsized":"2024-04-24T22:39:38.820279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705748+0000","last_change":"2024-04-24T22:39:37.703539+0000","last_active":"2024-04-24T22:39:38.705748+0000","last_peered":"2024-04-24T22:39:38.705748+0000","last_clean":"2024-04-24T22:39:38.705748+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:39:38.705748+0000","last_undegraded":"2024-04-24T22:39:38.705748+0000","last_fullsized":"2024-04-24T22:39:38.705748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707950+0000","last_change":"2024-04-24T22:39:37.713543+0000","last_active":"2024-04-24T22:39:38.707950+0000","last_peered":"2024-04-24T22:39:38.707950+0000","last_clean":"2024-04-24T22:39:38.707950+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:39:38.707950+0000","last_undegraded":"2024-04-24T22:39:38.707950+0000","last_fullsized":"2024-04-24T22:39:38.707950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706353+0000","last_change":"2024-04-24T22:39:37.706715+0000","last_active":"2024-04-24T22:39:38.706353+0000","last_peered":"2024-04-24T22:39:38.706353+0000","last_clean":"2024-04-24T22:39:38.706353+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:39:38.706353+0000","last_undegraded":"2024-04-24T22:39:38.706353+0000","last_fullsized":"2024-04-24T22:39:38.706353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819558+0000","last_change":"2024-04-24T22:39:37.701250+0000","last_active":"2024-04-24T22:39:38.819558+0000","last_peered":"2024-04-24T22:39:38.819558+0000","last_clean":"2024-04-24T22:39:38.819558+0000","last_became_active":"2024-04-24T22:39:37.700959+0000","last_became_peered":"2024-04-24T22:39:37.700959+0000","last_unstale":"2024-04-24T22:39:38.819558+0000","last_undegraded":"2024-04-24T22:39:38.819558+0000","last_fullsized":"2024-04-24T22:39:38.819558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819186+0000","last_change":"2024-04-24T22:39:37.701069+0000","last_active":"2024-04-24T22:39:38.819186+0000","last_peered":"2024-04-24T22:39:38.819186+0000","last_clean":"2024-04-24T22:39:38.819186+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:39:38.819186+0000","last_undegraded":"2024-04-24T22:39:38.819186+0000","last_fullsized":"2024-04-24T22:39:38.819186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819923+0000","last_change":"2024-04-24T22:39:37.693508+0000","last_active":"2024-04-24T22:39:38.819923+0000","last_peered":"2024-04-24T22:39:38.819923+0000","last_clean":"2024-04-24T22:39:38.819923+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:39:38.819923+0000","last_undegraded":"2024-04-24T22:39:38.819923+0000","last_fullsized":"2024-04-24T22:39:38.819923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708270+0000","last_change":"2024-04-24T22:39:37.705157+0000","last_active":"2024-04-24T22:39:38.708270+0000","last_peered":"2024-04-24T22:39:38.708270+0000","last_clean":"2024-04-24T22:39:38.708270+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:39:38.708270+0000","last_undegraded":"2024-04-24T22:39:38.708270+0000","last_fullsized":"2024-04-24T22:39:38.708270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704755+0000","last_change":"2024-04-24T22:39:37.698333+0000","last_active":"2024-04-24T22:39:38.704755+0000","last_peered":"2024-04-24T22:39:38.704755+0000","last_clean":"2024-04-24T22:39:38.704755+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:39:38.704755+0000","last_undegraded":"2024-04-24T22:39:38.704755+0000","last_fullsized":"2024-04-24T22:39:38.704755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:57.510197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820219+0000","last_change":"2024-04-24T22:39:37.696874+0000","last_active":"2024-04-24T22:39:38.820219+0000","last_peered":"2024-04-24T22:39:38.820219+0000","last_clean":"2024-04-24T22:39:38.820219+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:39:38.820219+0000","last_undegraded":"2024-04-24T22:39:38.820219+0000","last_fullsized":"2024-04-24T22:39:38.820219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820021+0000","last_change":"2024-04-24T22:39:37.703446+0000","last_active":"2024-04-24T22:39:38.820021+0000","last_peered":"2024-04-24T22:39:38.820021+0000","last_clean":"2024-04-24T22:39:38.820021+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:39:38.820021+0000","last_undegraded":"2024-04-24T22:39:38.820021+0000","last_fullsized":"2024-04-24T22:39:38.820021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820971+0000","last_change":"2024-04-24T22:39:37.701515+0000","last_active":"2024-04-24T22:39:38.820971+0000","last_peered":"2024-04-24T22:39:38.820971+0000","last_clean":"2024-04-24T22:39:38.820971+0000","last_became_active":"2024-04-24T22:39:37.700439+0000","last_became_peered":"2024-04-24T22:39:37.700439+0000","last_unstale":"2024-04-24T22:39:38.820971+0000","last_undegraded":"2024-04-24T22:39:38.820971+0000","last_fullsized":"2024-04-24T22:39:38.820971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819628+0000","last_change":"2024-04-24T22:39:37.703899+0000","last_active":"2024-04-24T22:39:38.819628+0000","last_peered":"2024-04-24T22:39:38.819628+0000","last_clean":"2024-04-24T22:39:38.819628+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:39:38.819628+0000","last_undegraded":"2024-04-24T22:39:38.819628+0000","last_fullsized":"2024-04-24T22:39:38.819628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706244+0000","last_change":"2024-04-24T22:39:37.711019+0000","last_active":"2024-04-24T22:39:38.706244+0000","last_peered":"2024-04-24T22:39:38.706244+0000","last_clean":"2024-04-24T22:39:38.706244+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:39:38.706244+0000","last_undegraded":"2024-04-24T22:39:38.706244+0000","last_fullsized":"2024-04-24T22:39:38.706244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819357+0000","last_change":"2024-04-24T22:39:37.696214+0000","last_active":"2024-04-24T22:39:38.819357+0000","last_peered":"2024-04-24T22:39:38.819357+0000","last_clean":"2024-04-24T22:39:38.819357+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:39:38.819357+0000","last_undegraded":"2024-04-24T22:39:38.819357+0000","last_fullsized":"2024-04-24T22:39:38.819357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705029+0000","last_change":"2024-04-24T22:39:37.700151+0000","last_active":"2024-04-24T22:39:38.705029+0000","last_peered":"2024-04-24T22:39:38.705029+0000","last_clean":"2024-04-24T22:39:38.705029+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:39:38.705029+0000","last_undegraded":"2024-04-24T22:39:38.705029+0000","last_fullsized":"2024-04-24T22:39:38.705029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820267+0000","last_change":"2024-04-24T22:39:37.705083+0000","last_active":"2024-04-24T22:39:38.820267+0000","last_peered":"2024-04-24T22:39:38.820267+0000","last_clean":"2024-04-24T22:39:38.820267+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:39:38.820267+0000","last_undegraded":"2024-04-24T22:39:38.820267+0000","last_fullsized":"2024-04-24T22:39:38.820267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708194+0000","last_change":"2024-04-24T22:39:37.711193+0000","last_active":"2024-04-24T22:39:38.708194+0000","last_peered":"2024-04-24T22:39:38.708194+0000","last_clean":"2024-04-24T22:39:38.708194+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:39:38.708194+0000","last_undegraded":"2024-04-24T22:39:38.708194+0000","last_fullsized":"2024-04-24T22:39:38.708194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820128+0000","last_change":"2024-04-24T22:39:37.713910+0000","last_active":"2024-04-24T22:39:38.820128+0000","last_peered":"2024-04-24T22:39:38.820128+0000","last_clean":"2024-04-24T22:39:38.820128+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:39:38.820128+0000","last_undegraded":"2024-04-24T22:39:38.820128+0000","last_fullsized":"2024-04-24T22:39:38.820128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708266+0000","last_change":"2024-04-24T22:39:37.717191+0000","last_active":"2024-04-24T22:39:38.708266+0000","last_peered":"2024-04-24T22:39:38.708266+0000","last_clean":"2024-04-24T22:39:38.708266+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:39:38.708266+0000","last_undegraded":"2024-04-24T22:39:38.708266+0000","last_fullsized":"2024-04-24T22:39:38.708266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819555+0000","last_change":"2024-04-24T22:39:37.702983+0000","last_active":"2024-04-24T22:39:38.819555+0000","last_peered":"2024-04-24T22:39:38.819555+0000","last_clean":"2024-04-24T22:39:38.819555+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:39:38.819555+0000","last_undegraded":"2024-04-24T22:39:38.819555+0000","last_fullsized":"2024-04-24T22:39:38.819555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820583+0000","last_change":"2024-04-24T22:39:37.714111+0000","last_active":"2024-04-24T22:39:38.820583+0000","last_peered":"2024-04-24T22:39:38.820583+0000","last_clean":"2024-04-24T22:39:38.820583+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:39:38.820583+0000","last_undegraded":"2024-04-24T22:39:38.820583+0000","last_fullsized":"2024-04-24T22:39:38.820583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708923+0000","last_change":"2024-04-24T22:39:37.714877+0000","last_active":"2024-04-24T22:39:38.708923+0000","last_peered":"2024-04-24T22:39:38.708923+0000","last_clean":"2024-04-24T22:39:38.708923+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:39:38.708923+0000","last_undegraded":"2024-04-24T22:39:38.708923+0000","last_fullsized":"2024-04-24T22:39:38.708923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705236+0000","last_change":"2024-04-24T22:39:37.714695+0000","last_active":"2024-04-24T22:39:38.705236+0000","last_peered":"2024-04-24T22:39:38.705236+0000","last_clean":"2024-04-24T22:39:38.705236+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:39:38.705236+0000","last_undegraded":"2024-04-24T22:39:38.705236+0000","last_fullsized":"2024-04-24T22:39:38.705236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820461+0000","last_change":"2024-04-24T22:39:37.719187+0000","last_active":"2024-04-24T22:39:38.820461+0000","last_peered":"2024-04-24T22:39:38.820461+0000","last_clean":"2024-04-24T22:39:38.820461+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:39:38.820461+0000","last_undegraded":"2024-04-24T22:39:38.820461+0000","last_fullsized":"2024-04-24T22:39:38.820461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819723+0000","last_change":"2024-04-24T22:39:37.707283+0000","last_active":"2024-04-24T22:39:38.819723+0000","last_peered":"2024-04-24T22:39:38.819723+0000","last_clean":"2024-04-24T22:39:38.819723+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:39:38.819723+0000","last_undegraded":"2024-04-24T22:39:38.819723+0000","last_fullsized":"2024-04-24T22:39:38.819723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818968+0000","last_change":"2024-04-24T22:39:37.697902+0000","last_active":"2024-04-24T22:39:38.818968+0000","last_peered":"2024-04-24T22:39:38.818968+0000","last_clean":"2024-04-24T22:39:38.818968+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:39:38.818968+0000","last_undegraded":"2024-04-24T22:39:38.818968+0000","last_fullsized":"2024-04-24T22:39:38.818968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819142+0000","last_change":"2024-04-24T22:39:37.705667+0000","last_active":"2024-04-24T22:39:38.819142+0000","last_peered":"2024-04-24T22:39:38.819142+0000","last_clean":"2024-04-24T22:39:38.819142+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:39:38.819142+0000","last_undegraded":"2024-04-24T22:39:38.819142+0000","last_fullsized":"2024-04-24T22:39:38.819142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:50.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:48.752812+0000","last_change":"2024-04-24T22:39:36.707990+0000","last_active":"2024-04-24T22:39:48.752812+0000","last_peered":"2024-04-24T22:39:48.752812+0000","last_clean":"2024-04-24T22:39:48.752812+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:39:48.752812+0000","last_undegraded":"2024-04-24T22:39:48.752812+0000","last_fullsized":"2024-04-24T22:39:48.752812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706420+0000","last_change":"2024-04-24T22:39:37.700406+0000","last_active":"2024-04-24T22:39:38.706420+0000","last_peered":"2024-04-24T22:39:38.706420+0000","last_clean":"2024-04-24T22:39:38.706420+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:39:38.706420+0000","last_undegraded":"2024-04-24T22:39:38.706420+0000","last_fullsized":"2024-04-24T22:39:38.706420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.821287+0000","last_change":"2024-04-24T22:39:36.701398+0000","last_active":"2024-04-24T22:39:59.821287+0000","last_peered":"2024-04-24T22:39:59.821287+0000","last_clean":"2024-04-24T22:39:59.821287+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:39:59.821287+0000","last_undegraded":"2024-04-24T22:39:59.821287+0000","last_fullsized":"2024-04-24T22:39:59.821287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:53.780440+0000","last_change":"2024-04-24T22:39:36.712294+0000","last_active":"2024-04-24T22:39:53.780440+0000","last_peered":"2024-04-24T22:39:53.780440+0000","last_clean":"2024-04-24T22:39:53.780440+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:39:53.780440+0000","last_undegraded":"2024-04-24T22:39:53.780440+0000","last_fullsized":"2024-04-24T22:39:53.780440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.707809+0000","last_change":"2024-04-24T22:39:37.716920+0000","last_active":"2024-04-24T22:39:38.707809+0000","last_peered":"2024-04-24T22:39:38.707809+0000","last_clean":"2024-04-24T22:39:38.707809+0000","last_became_active":"2024-04-24T22:39:37.716641+0000","last_became_peered":"2024-04-24T22:39:37.716641+0000","last_unstale":"2024-04-24T22:39:38.707809+0000","last_undegraded":"2024-04-24T22:39:38.707809+0000","last_fullsized":"2024-04-24T22:39:38.707809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.816798+0000","last_change":"2024-04-24T22:39:36.698983+0000","last_active":"2024-04-24T22:39:59.816798+0000","last_peered":"2024-04-24T22:39:59.816798+0000","last_clean":"2024-04-24T22:39:59.816798+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:39:59.816798+0000","last_undegraded":"2024-04-24T22:39:59.816798+0000","last_fullsized":"2024-04-24T22:39:59.816798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820135+0000","last_change":"2024-04-24T22:39:37.714229+0000","last_active":"2024-04-24T22:39:38.820135+0000","last_peered":"2024-04-24T22:39:38.820135+0000","last_clean":"2024-04-24T22:39:38.820135+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:39:38.820135+0000","last_undegraded":"2024-04-24T22:39:38.820135+0000","last_fullsized":"2024-04-24T22:39:38.820135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:48.752647+0000","last_change":"2024-04-24T22:39:36.705776+0000","last_active":"2024-04-24T22:39:48.752647+0000","last_peered":"2024-04-24T22:39:48.752647+0000","last_clean":"2024-04-24T22:39:48.752647+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:39:48.752647+0000","last_undegraded":"2024-04-24T22:39:48.752647+0000","last_fullsized":"2024-04-24T22:39:48.752647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820936+0000","last_change":"2024-04-24T22:39:37.701436+0000","last_active":"2024-04-24T22:39:38.820936+0000","last_peered":"2024-04-24T22:39:38.820936+0000","last_clean":"2024-04-24T22:39:38.820936+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:39:38.820936+0000","last_undegraded":"2024-04-24T22:39:38.820936+0000","last_fullsized":"2024-04-24T22:39:38.820936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.843014+0000","last_change":"2024-04-24T22:39:36.700237+0000","last_active":"2024-04-24T22:40:05.843014+0000","last_peered":"2024-04-24T22:40:05.843014+0000","last_clean":"2024-04-24T22:40:05.843014+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:40:05.843014+0000","last_undegraded":"2024-04-24T22:40:05.843014+0000","last_fullsized":"2024-04-24T22:40:05.843014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705984+0000","last_change":"2024-04-24T22:39:37.703532+0000","last_active":"2024-04-24T22:39:38.705984+0000","last_peered":"2024-04-24T22:39:38.705984+0000","last_clean":"2024-04-24T22:39:38.705984+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:39:38.705984+0000","last_undegraded":"2024-04-24T22:39:38.705984+0000","last_fullsized":"2024-04-24T22:39:38.705984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:48.750425+0000","last_change":"2024-04-24T22:39:36.717477+0000","last_active":"2024-04-24T22:39:48.750425+0000","last_peered":"2024-04-24T22:39:48.750425+0000","last_clean":"2024-04-24T22:39:48.750425+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:39:48.750425+0000","last_undegraded":"2024-04-24T22:39:48.750425+0000","last_fullsized":"2024-04-24T22:39:48.750425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819989+0000","last_change":"2024-04-24T22:39:37.699255+0000","last_active":"2024-04-24T22:39:38.819989+0000","last_peered":"2024-04-24T22:39:38.819989+0000","last_clean":"2024-04-24T22:39:38.819989+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:39:38.819989+0000","last_undegraded":"2024-04-24T22:39:38.819989+0000","last_fullsized":"2024-04-24T22:39:38.819989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'13","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:17.267156+0000","last_change":"2024-04-24T22:39:36.706567+0000","last_active":"2024-04-24T22:40:17.267156+0000","last_peered":"2024-04-24T22:40:17.267156+0000","last_clean":"2024-04-24T22:40:17.267156+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:40:17.267156+0000","last_undegraded":"2024-04-24T22:40:17.267156+0000","last_fullsized":"2024-04-24T22:40:17.267156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2685,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706118+0000","last_change":"2024-04-24T22:39:37.712064+0000","last_active":"2024-04-24T22:39:38.706118+0000","last_peered":"2024-04-24T22:39:38.706118+0000","last_clean":"2024-04-24T22:39:38.706118+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:39:38.706118+0000","last_undegraded":"2024-04-24T22:39:38.706118+0000","last_fullsized":"2024-04-24T22:39:38.706118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'9","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.842540+0000","last_change":"2024-04-24T22:39:36.700041+0000","last_active":"2024-04-24T22:40:05.842540+0000","last_peered":"2024-04-24T22:40:05.842540+0000","last_clean":"2024-04-24T22:40:05.842540+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:40:05.842540+0000","last_undegraded":"2024-04-24T22:40:05.842540+0000","last_fullsized":"2024-04-24T22:40:05.842540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:50.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705269+0000","last_change":"2024-04-24T22:39:37.691459+0000","last_active":"2024-04-24T22:39:38.705269+0000","last_peered":"2024-04-24T22:39:38.705269+0000","last_clean":"2024-04-24T22:39:38.705269+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:39:38.705269+0000","last_undegraded":"2024-04-24T22:39:38.705269+0000","last_fullsized":"2024-04-24T22:39:38.705269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.822296+0000","last_change":"2024-04-24T22:39:36.706454+0000","last_active":"2024-04-24T22:39:59.822296+0000","last_peered":"2024-04-24T22:39:59.822296+0000","last_clean":"2024-04-24T22:39:59.822296+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:39:59.822296+0000","last_undegraded":"2024-04-24T22:39:59.822296+0000","last_fullsized":"2024-04-24T22:39:59.822296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705898+0000","last_change":"2024-04-24T22:39:37.696724+0000","last_active":"2024-04-24T22:39:38.705898+0000","last_peered":"2024-04-24T22:39:38.705898+0000","last_clean":"2024-04-24T22:39:38.705898+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:39:38.705898+0000","last_undegraded":"2024-04-24T22:39:38.705898+0000","last_fullsized":"2024-04-24T22:39:38.705898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.820803+0000","last_change":"2024-04-24T22:39:36.701340+0000","last_active":"2024-04-24T22:39:59.820803+0000","last_peered":"2024-04-24T22:39:59.820803+0000","last_clean":"2024-04-24T22:39:59.820803+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:39:59.820803+0000","last_undegraded":"2024-04-24T22:39:59.820803+0000","last_fullsized":"2024-04-24T22:39:59.820803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819009+0000","last_change":"2024-04-24T22:39:37.697532+0000","last_active":"2024-04-24T22:39:38.819009+0000","last_peered":"2024-04-24T22:39:38.819009+0000","last_clean":"2024-04-24T22:39:38.819009+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:39:38.819009+0000","last_undegraded":"2024-04-24T22:39:38.819009+0000","last_fullsized":"2024-04-24T22:39:38.819009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'17","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.825168+0000","last_change":"2024-04-24T22:39:36.724649+0000","last_active":"2024-04-24T22:39:59.825168+0000","last_peered":"2024-04-24T22:39:59.825168+0000","last_clean":"2024-04-24T22:39:59.825168+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:39:59.825168+0000","last_undegraded":"2024-04-24T22:39:59.825168+0000","last_fullsized":"2024-04-24T22:39:59.825168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705839+0000","last_change":"2024-04-24T22:39:37.702890+0000","last_active":"2024-04-24T22:39:38.705839+0000","last_peered":"2024-04-24T22:39:38.705839+0000","last_clean":"2024-04-24T22:39:38.705839+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:39:38.705839+0000","last_undegraded":"2024-04-24T22:39:38.705839+0000","last_fullsized":"2024-04-24T22:39:38.705839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.720176+0000","last_change":"2024-04-24T22:39:36.716044+0000","last_active":"2024-04-24T22:39:42.720176+0000","last_peered":"2024-04-24T22:39:42.720176+0000","last_clean":"2024-04-24T22:39:42.720176+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:39:42.720176+0000","last_undegraded":"2024-04-24T22:39:42.720176+0000","last_fullsized":"2024-04-24T22:39:42.720176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:40.023933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706077+0000","last_change":"2024-04-24T22:39:37.702621+0000","last_active":"2024-04-24T22:39:38.706077+0000","last_peered":"2024-04-24T22:39:38.706077+0000","last_clean":"2024-04-24T22:39:38.706077+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:39:38.706077+0000","last_undegraded":"2024-04-24T22:39:38.706077+0000","last_fullsized":"2024-04-24T22:39:38.706077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.843760+0000","last_change":"2024-04-24T22:39:36.714272+0000","last_active":"2024-04-24T22:40:05.843760+0000","last_peered":"2024-04-24T22:40:05.843760+0000","last_clean":"2024-04-24T22:40:05.843760+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:40:05.843760+0000","last_undegraded":"2024-04-24T22:40:05.843760+0000","last_fullsized":"2024-04-24T22:40:05.843760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706373+0000","last_change":"2024-04-24T22:39:37.703786+0000","last_active":"2024-04-24T22:39:38.706373+0000","last_peered":"2024-04-24T22:39:38.706373+0000","last_clean":"2024-04-24T22:39:38.706373+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:39:38.706373+0000","last_undegraded":"2024-04-24T22:39:38.706373+0000","last_fullsized":"2024-04-24T22:39:38.706373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:55.990656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.823637+0000","last_change":"2024-04-24T22:39:36.715437+0000","last_active":"2024-04-24T22:39:59.823637+0000","last_peered":"2024-04-24T22:39:59.823637+0000","last_clean":"2024-04-24T22:39:59.823637+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:39:59.823637+0000","last_undegraded":"2024-04-24T22:39:59.823637+0000","last_fullsized":"2024-04-24T22:39:59.823637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820735+0000","last_change":"2024-04-24T22:39:37.715801+0000","last_active":"2024-04-24T22:39:38.820735+0000","last_peered":"2024-04-24T22:39:38.820735+0000","last_clean":"2024-04-24T22:39:38.820735+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:39:38.820735+0000","last_undegraded":"2024-04-24T22:39:38.820735+0000","last_fullsized":"2024-04-24T22:39:38.820735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.822281+0000","last_change":"2024-04-24T22:39:36.710422+0000","last_active":"2024-04-24T22:39:59.822281+0000","last_peered":"2024-04-24T22:39:59.822281+0000","last_clean":"2024-04-24T22:39:59.822281+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:39:59.822281+0000","last_undegraded":"2024-04-24T22:39:59.822281+0000","last_fullsized":"2024-04-24T22:39:59.822281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706031+0000","last_change":"2024-04-24T22:39:37.706745+0000","last_active":"2024-04-24T22:39:38.706031+0000","last_peered":"2024-04-24T22:39:38.706031+0000","last_clean":"2024-04-24T22:39:38.706031+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:39:38.706031+0000","last_undegraded":"2024-04-24T22:39:38.706031+0000","last_fullsized":"2024-04-24T22:39:38.706031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705671+0000","last_change":"2024-04-24T22:39:32.657965+0000","last_active":"2024-04-24T22:39:38.705671+0000","last_peered":"2024-04-24T22:39:38.705671+0000","last_clean":"2024-04-24T22:39:38.705671+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:39:38.705671+0000","last_undegraded":"2024-04-24T22:39:38.705671+0000","last_fullsized":"2024-04-24T22:39:38.705671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_clean_scrub_stamp":"2024-04-24T22:39:31.635804+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.714153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.822693+0000","last_change":"2024-04-24T22:39:36.710381+0000","last_active":"2024-04-24T22:39:59.822693+0000","last_peered":"2024-04-24T22:39:59.822693+0000","last_clean":"2024-04-24T22:39:59.822693+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:39:59.822693+0000","last_undegraded":"2024-04-24T22:39:59.822693+0000","last_fullsized":"2024-04-24T22:39:59.822693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708707+0000","last_change":"2024-04-24T22:39:37.713956+0000","last_active":"2024-04-24T22:39:38.708707+0000","last_peered":"2024-04-24T22:39:38.708707+0000","last_clean":"2024-04-24T22:39:38.708707+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:39:38.708707+0000","last_undegraded":"2024-04-24T22:39:38.708707+0000","last_fullsized":"2024-04-24T22:39:38.708707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:53.782609+0000","last_change":"2024-04-24T22:39:36.715290+0000","last_active":"2024-04-24T22:39:53.782609+0000","last_peered":"2024-04-24T22:39:53.782609+0000","last_clean":"2024-04-24T22:39:53.782609+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:39:53.782609+0000","last_undegraded":"2024-04-24T22:39:53.782609+0000","last_fullsized":"2024-04-24T22:39:53.782609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.704538+0000","last_change":"2024-04-24T22:39:37.695419+0000","last_active":"2024-04-24T22:39:38.704538+0000","last_peered":"2024-04-24T22:39:38.704538+0000","last_clean":"2024-04-24T22:39:38.704538+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:39:38.704538+0000","last_undegraded":"2024-04-24T22:39:38.704538+0000","last_fullsized":"2024-04-24T22:39:38.704538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:08.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:53.785443+0000","last_change":"2024-04-24T22:39:36.707548+0000","last_active":"2024-04-24T22:39:53.785443+0000","last_peered":"2024-04-24T22:39:53.785443+0000","last_clean":"2024-04-24T22:39:53.785443+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:39:53.785443+0000","last_undegraded":"2024-04-24T22:39:53.785443+0000","last_fullsized":"2024-04-24T22:39:53.785443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820375+0000","last_change":"2024-04-24T22:39:37.703301+0000","last_active":"2024-04-24T22:39:38.820375+0000","last_peered":"2024-04-24T22:39:38.820375+0000","last_clean":"2024-04-24T22:39:38.820375+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:39:38.820375+0000","last_undegraded":"2024-04-24T22:39:38.820375+0000","last_fullsized":"2024-04-24T22:39:38.820375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.846463+0000","last_change":"2024-04-24T22:39:36.713486+0000","last_active":"2024-04-24T22:40:05.846463+0000","last_peered":"2024-04-24T22:40:05.846463+0000","last_clean":"2024-04-24T22:40:05.846463+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:40:05.846463+0000","last_undegraded":"2024-04-24T22:40:05.846463+0000","last_fullsized":"2024-04-24T22:40:05.846463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819526+0000","last_change":"2024-04-24T22:39:37.705705+0000","last_active":"2024-04-24T22:39:38.819526+0000","last_peered":"2024-04-24T22:39:38.819526+0000","last_clean":"2024-04-24T22:39:38.819526+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:39:38.819526+0000","last_undegraded":"2024-04-24T22:39:38.819526+0000","last_fullsized":"2024-04-24T22:39:38.819526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:34.424877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:39.686949+0000","last_change":"2024-04-24T22:39:36.723125+0000","last_active":"2024-04-24T22:39:39.686949+0000","last_peered":"2024-04-24T22:39:39.686949+0000","last_clean":"2024-04-24T22:39:39.686949+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:39:39.686949+0000","last_undegraded":"2024-04-24T22:39:39.686949+0000","last_fullsized":"2024-04-24T22:39:39.686949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820922+0000","last_change":"2024-04-24T22:39:37.713947+0000","last_active":"2024-04-24T22:39:38.820922+0000","last_peered":"2024-04-24T22:39:38.820922+0000","last_clean":"2024-04-24T22:39:38.820922+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:39:38.820922+0000","last_undegraded":"2024-04-24T22:39:38.820922+0000","last_fullsized":"2024-04-24T22:39:38.820922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.821515+0000","last_change":"2024-04-24T22:39:36.716541+0000","last_active":"2024-04-24T22:39:59.821515+0000","last_peered":"2024-04-24T22:39:59.821515+0000","last_clean":"2024-04-24T22:39:59.821515+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:39:59.821515+0000","last_undegraded":"2024-04-24T22:39:59.821515+0000","last_fullsized":"2024-04-24T22:39:59.821515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:33.366773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820580+0000","last_change":"2024-04-24T22:39:37.708069+0000","last_active":"2024-04-24T22:39:38.820580+0000","last_peered":"2024-04-24T22:39:38.820580+0000","last_clean":"2024-04-24T22:39:38.820580+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:39:38.820580+0000","last_undegraded":"2024-04-24T22:39:38.820580+0000","last_fullsized":"2024-04-24T22:39:38.820580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.824116+0000","last_change":"2024-04-24T22:39:36.718740+0000","last_active":"2024-04-24T22:39:59.824116+0000","last_peered":"2024-04-24T22:39:59.824116+0000","last_clean":"2024-04-24T22:39:59.824116+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:39:59.824116+0000","last_undegraded":"2024-04-24T22:39:59.824116+0000","last_fullsized":"2024-04-24T22:39:59.824116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.709326+0000","last_change":"2024-04-24T22:39:37.704496+0000","last_active":"2024-04-24T22:39:38.709326+0000","last_peered":"2024-04-24T22:39:38.709326+0000","last_clean":"2024-04-24T22:39:38.709326+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:39:38.709326+0000","last_undegraded":"2024-04-24T22:39:38.709326+0000","last_fullsized":"2024-04-24T22:39:38.709326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.841962+0000","last_change":"2024-04-24T22:39:36.701308+0000","last_active":"2024-04-24T22:40:05.841962+0000","last_peered":"2024-04-24T22:40:05.841962+0000","last_clean":"2024-04-24T22:40:05.841962+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:40:05.841962+0000","last_undegraded":"2024-04-24T22:40:05.841962+0000","last_fullsized":"2024-04-24T22:40:05.841962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.705483+0000","last_change":"2024-04-24T22:39:37.700189+0000","last_active":"2024-04-24T22:39:38.705483+0000","last_peered":"2024-04-24T22:39:38.705483+0000","last_clean":"2024-04-24T22:39:38.705483+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:39:38.705483+0000","last_undegraded":"2024-04-24T22:39:38.705483+0000","last_fullsized":"2024-04-24T22:39:38.705483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:48.749442+0000","last_change":"2024-04-24T22:39:36.716460+0000","last_active":"2024-04-24T22:39:48.749442+0000","last_peered":"2024-04-24T22:39:48.749442+0000","last_clean":"2024-04-24T22:39:48.749442+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:39:48.749442+0000","last_undegraded":"2024-04-24T22:39:48.749442+0000","last_fullsized":"2024-04-24T22:39:48.749442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.819020+0000","last_change":"2024-04-24T22:39:37.704143+0000","last_active":"2024-04-24T22:39:38.819020+0000","last_peered":"2024-04-24T22:39:38.819020+0000","last_clean":"2024-04-24T22:39:38.819020+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:39:38.819020+0000","last_undegraded":"2024-04-24T22:39:38.819020+0000","last_fullsized":"2024-04-24T22:39:38.819020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:58.556519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.844145+0000","last_change":"2024-04-24T22:39:36.721540+0000","last_active":"2024-04-24T22:40:05.844145+0000","last_peered":"2024-04-24T22:40:05.844145+0000","last_clean":"2024-04-24T22:40:05.844145+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:40:05.844145+0000","last_undegraded":"2024-04-24T22:40:05.844145+0000","last_fullsized":"2024-04-24T22:40:05.844145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708008+0000","last_change":"2024-04-24T22:39:37.714150+0000","last_active":"2024-04-24T22:39:38.708008+0000","last_peered":"2024-04-24T22:39:38.708008+0000","last_clean":"2024-04-24T22:39:38.708008+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:39:38.708008+0000","last_undegraded":"2024-04-24T22:39:38.708008+0000","last_fullsized":"2024-04-24T22:39:38.708008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:59.823570+0000","last_change":"2024-04-24T22:39:36.716667+0000","last_active":"2024-04-24T22:39:59.823570+0000","last_peered":"2024-04-24T22:39:59.823570+0000","last_clean":"2024-04-24T22:39:59.823570+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:39:59.823570+0000","last_undegraded":"2024-04-24T22:39:59.823570+0000","last_fullsized":"2024-04-24T22:39:59.823570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708586+0000","last_change":"2024-04-24T22:39:37.715906+0000","last_active":"2024-04-24T22:39:38.708586+0000","last_peered":"2024-04-24T22:39:38.708586+0000","last_clean":"2024-04-24T22:39:38.708586+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:39:38.708586+0000","last_undegraded":"2024-04-24T22:39:38.708586+0000","last_fullsized":"2024-04-24T22:39:38.708586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:48.748623+0000","last_change":"2024-04-24T22:39:36.698212+0000","last_active":"2024-04-24T22:39:48.748623+0000","last_peered":"2024-04-24T22:39:48.748623+0000","last_clean":"2024-04-24T22:39:48.748623+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:39:48.748623+0000","last_undegraded":"2024-04-24T22:39:48.748623+0000","last_fullsized":"2024-04-24T22:39:48.748623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.818963+0000","last_change":"2024-04-24T22:39:37.696143+0000","last_active":"2024-04-24T22:39:38.818963+0000","last_peered":"2024-04-24T22:39:38.818963+0000","last_clean":"2024-04-24T22:39:38.818963+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:39:38.818963+0000","last_undegraded":"2024-04-24T22:39:38.818963+0000","last_fullsized":"2024-04-24T22:39:38.818963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820834+0000","last_change":"2024-04-24T22:39:37.696113+0000","last_active":"2024-04-24T22:39:38.820834+0000","last_peered":"2024-04-24T22:39:38.820834+0000","last_clean":"2024-04-24T22:39:38.820834+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:39:38.820834+0000","last_undegraded":"2024-04-24T22:39:38.820834+0000","last_fullsized":"2024-04-24T22:39:38.820834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.842425+0000","last_change":"2024-04-24T22:39:36.699149+0000","last_active":"2024-04-24T22:40:05.842425+0000","last_peered":"2024-04-24T22:40:05.842425+0000","last_clean":"2024-04-24T22:40:05.842425+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:40:05.842425+0000","last_undegraded":"2024-04-24T22:40:05.842425+0000","last_fullsized":"2024-04-24T22:40:05.842425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.820550+0000","last_change":"2024-04-24T22:39:37.700972+0000","last_active":"2024-04-24T22:39:38.820550+0000","last_peered":"2024-04-24T22:39:38.820550+0000","last_clean":"2024-04-24T22:39:38.820550+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:39:38.820550+0000","last_undegraded":"2024-04-24T22:39:38.820550+0000","last_fullsized":"2024-04-24T22:39:38.820550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'12","reported_seq":26,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.843035+0000","last_change":"2024-04-24T22:39:36.698567+0000","last_active":"2024-04-24T22:40:05.843035+0000","last_peered":"2024-04-24T22:40:05.843035+0000","last_clean":"2024-04-24T22:40:05.843035+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:40:05.843035+0000","last_undegraded":"2024-04-24T22:40:05.843035+0000","last_fullsized":"2024-04-24T22:40:05.843035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.706000+0000","last_change":"2024-04-24T22:39:37.706951+0000","last_active":"2024-04-24T22:39:38.706000+0000","last_peered":"2024-04-24T22:39:38.706000+0000","last_clean":"2024-04-24T22:39:38.706000+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:39:38.706000+0000","last_undegraded":"2024-04-24T22:39:38.706000+0000","last_fullsized":"2024-04-24T22:39:38.706000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:42.729606+0000","last_change":"2024-04-24T22:39:36.703823+0000","last_active":"2024-04-24T22:39:42.729606+0000","last_peered":"2024-04-24T22:39:42.729606+0000","last_clean":"2024-04-24T22:39:42.729606+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:39:42.729606+0000","last_undegraded":"2024-04-24T22:39:42.729606+0000","last_fullsized":"2024-04-24T22:39:42.729606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:39:38.708601+0000","last_change":"2024-04-24T22:39:37.712078+0000","last_active":"2024-04-24T22:39:38.708601+0000","last_peered":"2024-04-24T22:39:38.708601+0000","last_clean":"2024-04-24T22:39:38.708601+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:39:38.708601+0000","last_undegraded":"2024-04-24T22:39:38.708601+0000","last_fullsized":"2024-04-24T22:39:38.708601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'13","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-24T22:40:05.843115+0000","last_change":"2024-04-24T22:39:36.719643+0000","last_active":"2024-04-24T22:40:05.843115+0000","last_peered":"2024-04-24T22:40:05.843115+0000","last_clean":"2024-04-24T22:40:05.843115+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:40:05.843115+0000","last_undegraded":"2024-04-24T22:40:05.843115+0000","last_fullsized":"2024-04-24T22:40:05.843115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9365,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":209,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":18730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":847872,"data_stored":668073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":79080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":79795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":76935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":249856,"data_stored":79114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":249856,"data_stored":77067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":245760,"data_stored":77395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":839680,"data_stored":666847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:40:20.908 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-04-24T22:40:20.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:40:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.162+0000 7fd439000640 1 Processor -- start 2024-04-24T22:40:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.162+0000 7fd439000640 1 -- start start 2024-04-24T22:40:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.163+0000 7fd439000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8b30 0x7fd434154520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.163+0000 7fd439000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd43415a730 con 0x7fd43415e8c0 2024-04-24T22:40:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.163+0000 7fd439000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd43415a9b0 con 0x7fd4340c8f10 2024-04-24T22:40:21.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.163+0000 7fd439000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd43406c730 con 0x7fd4340c8b30 2024-04-24T22:40:21.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.164+0000 7fd433e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd43415e8c0 0x7fd434066f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56520/0 (socket says 172.21.15.47:56520) 2024-04-24T22:40:21.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.164+0000 7fd433e00640 1 -- 172.21.15.47:0/819494947 learned_addr learned my addr 172.21.15.47:0/819494947 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:21.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.164+0000 7fd433400640 1 --2- 172.21.15.47:0/819494947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8b30 0x7fd434154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.164+0000 7fd432000640 1 -- 172.21.15.47:0/819494947 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122842696 0 0) 0x7fd43415a9b0 con 0x7fd4340c8f10 2024-04-24T22:40:21.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.164+0000 7fd432000640 1 -- 172.21.15.47:0/819494947 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd418003520 con 0x7fd4340c8f10 2024-04-24T22:40:21.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.164+0000 7fd432000640 1 -- 172.21.15.47:0/819494947 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342665504 0 0) 0x7fd42c001490 con 0x7fd43415e8c0 2024-04-24T22:40:21.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.164+0000 7fd433400640 1 -- 172.21.15.47:0/819494947 >> v1:172.21.15.113:6789/0 conn(0x7fd4340c8f10 legacy=0x7fd43415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.164+0000 7fd433400640 1 -- 172.21.15.47:0/819494947 >> v1:172.21.15.47:6789/0 conn(0x7fd43415e8c0 legacy=0x7fd434066f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.164+0000 7fd433400640 1 -- 172.21.15.47:0/819494947 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43415ac90 con 0x7fd4340c8b30 2024-04-24T22:40:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.165+0000 7fd433400640 1 --2- 172.21.15.47:0/819494947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8b30 0x7fd434154520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4240097d0 tx=0x7fd42402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b61e0f00068404d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.165+0000 7fd432000640 1 -- 172.21.15.47:0/819494947 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424037070 con 0x7fd4340c8b30 2024-04-24T22:40:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.166+0000 7fd432000640 1 -- 172.21.15.47:0/819494947 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd424032440 con 0x7fd4340c8b30 2024-04-24T22:40:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.166+0000 7fd432000640 1 -- 172.21.15.47:0/819494947 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4240327f0 con 0x7fd4340c8b30 2024-04-24T22:40:21.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.166+0000 7fd439000640 1 -- 172.21.15.47:0/819494947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8b30 msgr2=0x7fd434154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.166+0000 7fd439000640 1 --2- 172.21.15.47:0/819494947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8b30 0x7fd434154520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4240097d0 tx=0x7fd42402a210 comp rx=0 tx=0).stop 2024-04-24T22:40:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.166+0000 7fd439000640 1 -- 172.21.15.47:0/819494947 shutdown_connections 2024-04-24T22:40:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.166+0000 7fd439000640 1 --2- 172.21.15.47:0/819494947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8b30 0x7fd434154520 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.166+0000 7fd439000640 1 -- 172.21.15.47:0/819494947 >> 172.21.15.47:0/819494947 conn(0x7fd4340c9460 msgr2=0x7fd4340c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.166+0000 7fd439000640 1 -- 172.21.15.47:0/819494947 shutdown_connections 2024-04-24T22:40:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.166+0000 7fd439000640 1 -- 172.21.15.47:0/819494947 wait complete. 2024-04-24T22:40:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.168+0000 7fd439000640 1 Processor -- start 2024-04-24T22:40:21.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.168+0000 7fd439000640 1 -- start start 2024-04-24T22:40:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd439000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4340c8b30 0x7fd43414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd439000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8f10 0x7fd434151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd439000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd43415e8c0 0x7fd434151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd439000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd43415b530 con 0x7fd43415e8c0 2024-04-24T22:40:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd439000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd43415b250 con 0x7fd4340c8b30 2024-04-24T22:40:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd439000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd434154a60 con 0x7fd4340c8f10 2024-04-24T22:40:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd433e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd43415e8c0 0x7fd434151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd432a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8f10 0x7fd434151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd433400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4340c8b30 0x7fd43414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd432a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8f10 0x7fd434151770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34386/0 (socket says 172.21.15.47:34386) 2024-04-24T22:40:21.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd433e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd43415e8c0 0x7fd434151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60546/0 (socket says 172.21.15.47:60546) 2024-04-24T22:40:21.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.169+0000 7fd432a00640 1 -- 172.21.15.47:0/1133185700 learned_addr learned my addr 172.21.15.47:0/1133185700 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.170+0000 7fd432a00640 1 -- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4340c8b30 msgr2=0x7fd43414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.170+0000 7fd432a00640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4340c8b30 0x7fd43414bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.170+0000 7fd432a00640 1 -- 172.21.15.47:0/1133185700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd43415e8c0 msgr2=0x7fd434151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.170+0000 7fd432a00640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd43415e8c0 0x7fd434151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.170+0000 7fd432a00640 1 -- 172.21.15.47:0/1133185700 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4340c5240 con 0x7fd4340c8f10 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.170+0000 7fd432a00640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8f10 0x7fd434151770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd42000c530 tx=0x7fd42000c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.170+0000 7fd42b400640 1 -- 172.21.15.47:0/1133185700 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4200052d0 con 0x7fd4340c8f10 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.170+0000 7fd42b400640 1 -- 172.21.15.47:0/1133185700 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd420005490 con 0x7fd4340c8f10 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.171+0000 7fd439000640 1 -- 172.21.15.47:0/1133185700 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43415d8e0 con 0x7fd4340c8f10 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.171+0000 7fd42b400640 1 -- 172.21.15.47:0/1133185700 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420012870 con 0x7fd4340c8f10 2024-04-24T22:40:21.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.171+0000 7fd439000640 1 -- 172.21.15.47:0/1133185700 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4340c5ab0 con 0x7fd4340c8f10 2024-04-24T22:40:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.172+0000 7fd439000640 1 -- 172.21.15.47:0/1133185700 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43404f5e0 con 0x7fd4340c8f10 2024-04-24T22:40:21.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.173+0000 7fd42b400640 1 -- 172.21.15.47:0/1133185700 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd420012a10 con 0x7fd4340c8f10 2024-04-24T22:40:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.174+0000 7fd42b400640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd41406be30 0x7fd41406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.174+0000 7fd42b400640 1 -- 172.21.15.47:0/1133185700 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd42008aa10 con 0x7fd4340c8f10 2024-04-24T22:40:21.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.174+0000 7fd433400640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd41406be30 0x7fd41406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.175+0000 7fd433400640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd41406be30 0x7fd41406e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd424034000 tx=0x7fd42402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:21.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.178+0000 7fd42b400640 1 -- 172.21.15.47:0/1133185700 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd420058ce0 con 0x7fd4340c8f10 2024-04-24T22:40:21.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.384+0000 7fd439000640 1 -- 172.21.15.47:0/1133185700 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd43406c2f0 con 0x7fd4340c8f10 2024-04-24T22:40:21.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.386+0000 7fd42b400640 1 -- 172.21.15.47:0/1133185700 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16593 (secure 0 0 0) 0x7fd42005cfe0 con 0x7fd4340c8f10 2024-04-24T22:40:21.388 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:21.389 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:39:38.692613+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 -- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd41406be30 msgr2=0x7fd41406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd41406be30 0x7fd41406e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd424034000 tx=0x7fd42402e000 comp rx=0 tx=0).stop 2024-04-24T22:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 -- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8f10 msgr2=0x7fd434151770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8f10 0x7fd434151770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd42000c530 tx=0x7fd42000c9f0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 -- 172.21.15.47:0/1133185700 shutdown_connections 2024-04-24T22:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd41406be30 0x7fd41406e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd43415e8c0 0x7fd434151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4340c8f10 0x7fd434151770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 --2- 172.21.15.47:0/1133185700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4340c8b30 0x7fd43414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 -- 172.21.15.47:0/1133185700 >> 172.21.15.47:0/1133185700 conn(0x7fd4340c9460 msgr2=0x7fd434159ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 -- 172.21.15.47:0/1133185700 shutdown_connections 2024-04-24T22:40:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.389+0000 7fd439000640 1 -- 172.21.15.47:0/1133185700 wait complete. 2024-04-24T22:40:21.410 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:30.711538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:34.803185+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:36.184123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:40:21.410 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:40:21.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.616+0000 7f6e9ae00640 1 Processor -- start 2024-04-24T22:40:21.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.616+0000 7f6e9ae00640 1 -- start start 2024-04-24T22:40:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.618+0000 7f6e9ae00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6e940c1a00 0x7f6e94072f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.618+0000 7f6e9ae00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940734a0 0x7f6e941576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.618+0000 7f6e9ae00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e9415aa30 con 0x7f6e94151780 2024-04-24T22:40:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.618+0000 7f6e9ae00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6e94073930 con 0x7f6e940c1a00 2024-04-24T22:40:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.618+0000 7f6e9ae00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9415ad10 con 0x7f6e940734a0 2024-04-24T22:40:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.618+0000 7f6e93e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6e94151780 0x7f6e940c8590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56532/0 (socket says 172.21.15.47:56532) 2024-04-24T22:40:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.618+0000 7f6e93e00640 1 -- 172.21.15.47:0/2572688295 learned_addr learned my addr 172.21.15.47:0/2572688295 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.619+0000 7f6e98e00640 1 --2- 172.21.15.47:0/2572688295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940734a0 0x7f6e941576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.619+0000 7f6e8be00640 1 --2- 172.21.15.47:0/2572688295 >> v2:172.21.15.47:3300/0 conn(0x7f6e940c1a00 0x7f6e94072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.619+0000 7f6e93400640 1 -- 172.21.15.47:0/2572688295 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983472055 0 0) 0x7f6e9415aa30 con 0x7f6e94151780 2024-04-24T22:40:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.619+0000 7f6e93400640 1 -- 172.21.15.47:0/2572688295 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e78003520 con 0x7f6e94151780 2024-04-24T22:40:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.619+0000 7f6e98e00640 1 -- 172.21.15.47:0/2572688295 >> v2:172.21.15.47:3300/0 conn(0x7f6e940c1a00 msgr2=0x7f6e94072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.619+0000 7f6e98e00640 1 --2- 172.21.15.47:0/2572688295 >> v2:172.21.15.47:3300/0 conn(0x7f6e940c1a00 0x7f6e94072f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.619+0000 7f6e98e00640 1 -- 172.21.15.47:0/2572688295 >> v1:172.21.15.47:6789/0 conn(0x7f6e94151780 legacy=0x7f6e940c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.619+0000 7f6e98e00640 1 -- 172.21.15.47:0/2572688295 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9415aff0 con 0x7f6e940734a0 2024-04-24T22:40:21.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.620+0000 7f6e98e00640 1 --2- 172.21.15.47:0/2572688295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940734a0 0x7f6e941576f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009d90 tx=0x7f6e8c02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6065399dac09a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:21.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.621+0000 7f6e93400640 1 -- 172.21.15.47:0/2572688295 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c007740 con 0x7f6e940734a0 2024-04-24T22:40:21.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.621+0000 7f6e93400640 1 -- 172.21.15.47:0/2572688295 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e8c0078e0 con 0x7f6e940734a0 2024-04-24T22:40:21.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.621+0000 7f6e93400640 1 -- 172.21.15.47:0/2572688295 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c007cb0 con 0x7f6e940734a0 2024-04-24T22:40:21.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.621+0000 7f6e9ae00640 1 -- 172.21.15.47:0/2572688295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940734a0 msgr2=0x7f6e941576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.621+0000 7f6e9ae00640 1 --2- 172.21.15.47:0/2572688295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940734a0 0x7f6e941576f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009d90 tx=0x7f6e8c02a6f0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.621+0000 7f6e9ae00640 1 -- 172.21.15.47:0/2572688295 shutdown_connections 2024-04-24T22:40:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.621+0000 7f6e9ae00640 1 --2- 172.21.15.47:0/2572688295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940734a0 0x7f6e941576f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.621+0000 7f6e9ae00640 1 --2- 172.21.15.47:0/2572688295 >> v2:172.21.15.47:3300/0 conn(0x7f6e940c1a00 0x7f6e94072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.621+0000 7f6e9ae00640 1 -- 172.21.15.47:0/2572688295 >> 172.21.15.47:0/2572688295 conn(0x7f6e9406e580 msgr2=0x7f6e9406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.621+0000 7f6e9ae00640 1 -- 172.21.15.47:0/2572688295 shutdown_connections 2024-04-24T22:40:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.622+0000 7f6e9ae00640 1 -- 172.21.15.47:0/2572688295 wait complete. 2024-04-24T22:40:21.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.623+0000 7f6e9ae00640 1 Processor -- start 2024-04-24T22:40:21.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.624+0000 7f6e9ae00640 1 -- start start 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.624+0000 7f6e9ae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e940734a0 0x7f6e94147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.624+0000 7f6e9ae00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940c1a00 0x7f6e941e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e9ae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e94151780 0x7f6e941e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e9ae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e940c8ca0 con 0x7f6e940734a0 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e9ae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9415b5b0 con 0x7f6e94151780 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e9ae00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6e940c8e20 con 0x7f6e940c1a00 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e93e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e940734a0 0x7f6e94147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e93e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e940734a0 0x7f6e94147110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60568/0 (socket says 172.21.15.47:60568) 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e93e00640 1 -- 172.21.15.47:0/1062601691 learned_addr learned my addr 172.21.15.47:0/1062601691 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e8be00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940c1a00 0x7f6e941e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e98e00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e94151780 0x7f6e941e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e93e00640 1 -- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940c1a00 msgr2=0x7f6e941e5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e93e00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940c1a00 0x7f6e941e5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e93e00640 1 -- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e94151780 msgr2=0x7f6e941e5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e93e00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e94151780 0x7f6e941e5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.625+0000 7f6e93e00640 1 -- 172.21.15.47:0/1062601691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e94147650 con 0x7f6e940734a0 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.626+0000 7f6e93e00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e940734a0 0x7f6e94147110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c460 tx=0x7f6e8400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.626+0000 7f6e90c00640 1 -- 172.21.15.47:0/1062601691 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84017070 con 0x7f6e940734a0 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.626+0000 7f6e90c00640 1 -- 172.21.15.47:0/1062601691 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e84005ce0 con 0x7f6e940734a0 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.626+0000 7f6e90c00640 1 -- 172.21.15.47:0/1062601691 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84007290 con 0x7f6e940734a0 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.626+0000 7f6e9ae00640 1 -- 172.21.15.47:0/1062601691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e941f0390 con 0x7f6e940734a0 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.626+0000 7f6e9ae00640 1 -- 172.21.15.47:0/1062601691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e941f0950 con 0x7f6e940734a0 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.627+0000 7f6e9ae00640 1 -- 172.21.15.47:0/1062601691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e9415aa30 con 0x7f6e940734a0 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.628+0000 7f6e90c00640 1 -- 172.21.15.47:0/1062601691 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e8401a070 con 0x7f6e940734a0 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.629+0000 7f6e90c00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7006be30 0x7f6e7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.629+0000 7f6e90c00640 1 -- 172.21.15.47:0/1062601691 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6e8408a210 con 0x7f6e940734a0 2024-04-24T22:40:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.630+0000 7f6e8be00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7006be30 0x7f6e7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.630+0000 7f6e8be00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7006be30 0x7f6e7006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e941580d0 tx=0x7f6e7c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.634+0000 7f6e90c00640 1 -- 172.21.15.47:0/1062601691 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e9415aa30 con 0x7f6e940734a0 2024-04-24T22:40:21.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.831+0000 7f6e9ae00640 1 -- 172.21.15.47:0/1062601691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e94073b90 con 0x7f6e940734a0 2024-04-24T22:40:21.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.832+0000 7f6e90c00640 1 -- 172.21.15.47:0/1062601691 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f6e840584d0 con 0x7f6e940734a0 2024-04-24T22:40:21.834 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-24T22:40:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.834+0000 7f6e9ae00640 1 -- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7006be30 msgr2=0x7f6e7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.834+0000 7f6e9ae00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7006be30 0x7f6e7006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e941580d0 tx=0x7f6e7c009190 comp rx=0 tx=0).stop 2024-04-24T22:40:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.834+0000 7f6e9ae00640 1 -- 172.21.15.47:0/1062601691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e940734a0 msgr2=0x7f6e94147110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.834+0000 7f6e9ae00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e940734a0 0x7f6e94147110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c460 tx=0x7f6e8400c920 comp rx=0 tx=0).stop 2024-04-24T22:40:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.835+0000 7f6e9ae00640 1 -- 172.21.15.47:0/1062601691 shutdown_connections 2024-04-24T22:40:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.835+0000 7f6e9ae00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6e7006be30 0x7f6e7006e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.835+0000 7f6e9ae00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e94151780 0x7f6e941e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.835+0000 7f6e9ae00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e940c1a00 0x7f6e941e5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.835+0000 7f6e9ae00640 1 --2- 172.21.15.47:0/1062601691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e940734a0 0x7f6e94147110 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:21.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.835+0000 7f6e9ae00640 1 -- 172.21.15.47:0/1062601691 >> 172.21.15.47:0/1062601691 conn(0x7f6e9406e580 msgr2=0x7f6e941575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:21.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.835+0000 7f6e9ae00640 1 -- 172.21.15.47:0/1062601691 shutdown_connections 2024-04-24T22:40:21.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:21.835+0000 7f6e9ae00640 1 -- 172.21.15.47:0/1062601691 wait complete. 2024-04-24T22:40:21.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:40:22.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.056+0000 7f6510c00640 1 Processor -- start 2024-04-24T22:40:22.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.056+0000 7f6510c00640 1 -- start start 2024-04-24T22:40:22.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.058+0000 7f6510c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f650c073040 0x7f650c1411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.058+0000 7f6510c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f650c15c3d0 con 0x7f650c06bb90 2024-04-24T22:40:22.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.058+0000 7f6510c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f650c15c6b0 con 0x7f650c072c60 2024-04-24T22:40:22.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.058+0000 7f6510c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f650c158bf0 con 0x7f650c073040 2024-04-24T22:40:22.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.058+0000 7f650aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f650c073040 0x7f650c1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.058+0000 7f650b400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f650c072c60 0x7f650c0c2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:59592/0 (socket says 172.21.15.47:59592) 2024-04-24T22:40:22.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.058+0000 7f650b400640 1 -- 172.21.15.47:0/745659036 learned_addr learned my addr 172.21.15.47:0/745659036 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:22.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.059+0000 7f650aa00640 1 -- 172.21.15.47:0/745659036 >> v1:172.21.15.113:6789/0 conn(0x7f650c072c60 legacy=0x7f650c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.059+0000 7f650aa00640 1 -- 172.21.15.47:0/745659036 >> v1:172.21.15.47:6789/0 conn(0x7f650c06bb90 legacy=0x7f650c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.059+0000 7f650aa00640 1 -- 172.21.15.47:0/745659036 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c1585a0 con 0x7f650c073040 2024-04-24T22:40:22.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.060+0000 7f650aa00640 1 --2- 172.21.15.47:0/745659036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f650c073040 0x7f650c1411f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f64f80097a0 tx=0x7f64f802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c6d86d283a87839 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:22.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.060+0000 7f650a000640 1 -- 172.21.15.47:0/745659036 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8037070 con 0x7f650c073040 2024-04-24T22:40:22.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.060+0000 7f650a000640 1 -- 172.21.15.47:0/745659036 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f8032440 con 0x7f650c073040 2024-04-24T22:40:22.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.060+0000 7f650a000640 1 -- 172.21.15.47:0/745659036 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8032810 con 0x7f650c073040 2024-04-24T22:40:22.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.061+0000 7f6510c00640 1 -- 172.21.15.47:0/745659036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f650c073040 msgr2=0x7f650c1411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.061+0000 7f6510c00640 1 --2- 172.21.15.47:0/745659036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f650c073040 0x7f650c1411f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f64f80097a0 tx=0x7f64f802a230 comp rx=0 tx=0).stop 2024-04-24T22:40:22.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.061+0000 7f6510c00640 1 -- 172.21.15.47:0/745659036 shutdown_connections 2024-04-24T22:40:22.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.061+0000 7f6510c00640 1 --2- 172.21.15.47:0/745659036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f650c073040 0x7f650c1411f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.061+0000 7f6510c00640 1 -- 172.21.15.47:0/745659036 >> 172.21.15.47:0/745659036 conn(0x7f650c0cb900 msgr2=0x7f650c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:22.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.061+0000 7f6510c00640 1 -- 172.21.15.47:0/745659036 shutdown_connections 2024-04-24T22:40:22.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.061+0000 7f6510c00640 1 -- 172.21.15.47:0/745659036 wait complete. 2024-04-24T22:40:22.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.063+0000 7f6510c00640 1 Processor -- start 2024-04-24T22:40:22.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.063+0000 7f6510c00640 1 -- start start 2024-04-24T22:40:22.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.064+0000 7f6510c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650c06bb90 0x7f650c1efd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.064+0000 7f6510c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f650c072c60 0x7f650c1ec220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.064+0000 7f6510c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f650c073040 0x7f650c1ec760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.064+0000 7f6510c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f650c159080 con 0x7f650c06bb90 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.064+0000 7f6510c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f650c158270 con 0x7f650c073040 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.064+0000 7f6510c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f650c159200 con 0x7f650c072c60 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.065+0000 7f650b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650c06bb90 0x7f650c1efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.065+0000 7f650b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650c06bb90 0x7f650c1efd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60582/0 (socket says 172.21.15.47:60582) 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.065+0000 7f650b400640 1 -- 172.21.15.47:0/2838004220 learned_addr learned my addr 172.21.15.47:0/2838004220 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.065+0000 7f650be00640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f650c073040 0x7f650c1ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.065+0000 7f650aa00640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f650c072c60 0x7f650c1ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.065+0000 7f650b400640 1 -- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f650c072c60 msgr2=0x7f650c1ec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.065+0000 7f650b400640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f650c072c60 0x7f650c1ec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.065+0000 7f650b400640 1 -- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f650c073040 msgr2=0x7f650c1ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.065+0000 7f650b400640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f650c073040 0x7f650c1ec760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.065+0000 7f650b400640 1 -- 172.21.15.47:0/2838004220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c1f03e0 con 0x7f650c06bb90 2024-04-24T22:40:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.066+0000 7f650b400640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650c06bb90 0x7f650c1efd50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c4a0 tx=0x7f64fc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.066+0000 7f6503400640 1 -- 172.21.15.47:0/2838004220 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc017070 con 0x7f650c06bb90 2024-04-24T22:40:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.066+0000 7f6503400640 1 -- 172.21.15.47:0/2838004220 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64fc005ce0 con 0x7f650c06bb90 2024-04-24T22:40:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.066+0000 7f6503400640 1 -- 172.21.15.47:0/2838004220 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc007270 con 0x7f650c06bb90 2024-04-24T22:40:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.066+0000 7f6510c00640 1 -- 172.21.15.47:0/2838004220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c1f06c0 con 0x7f650c06bb90 2024-04-24T22:40:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.066+0000 7f6510c00640 1 -- 172.21.15.47:0/2838004220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c1f0c80 con 0x7f650c06bb90 2024-04-24T22:40:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.067+0000 7f6510c00640 1 -- 172.21.15.47:0/2838004220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650c068d50 con 0x7f650c06bb90 2024-04-24T22:40:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.068+0000 7f6503400640 1 -- 172.21.15.47:0/2838004220 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64fc01a070 con 0x7f650c06bb90 2024-04-24T22:40:22.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.069+0000 7f6503400640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06be30 0x7f64ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.069+0000 7f6503400640 1 -- 172.21.15.47:0/2838004220 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f64fc08a1a0 con 0x7f650c06bb90 2024-04-24T22:40:22.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.070+0000 7f650aa00640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06be30 0x7f64ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.070+0000 7f650aa00640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06be30 0x7f64ec06e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f64f8034000 tx=0x7f64f802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:22.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.074+0000 7f6503400640 1 -- 172.21.15.47:0/2838004220 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f64fc058460 con 0x7f650c06bb90 2024-04-24T22:40:22.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.281+0000 7f6510c00640 1 -- 172.21.15.47:0/2838004220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f650c04f5e0 con 0x7f650c06bb90 2024-04-24T22:40:22.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.282+0000 7f6503400640 1 -- 172.21.15.47:0/2838004220 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f64fc05c760 con 0x7f650c06bb90 2024-04-24T22:40:22.284 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-24T22:40:22.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 -- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06be30 msgr2=0x7f64ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06be30 0x7f64ec06e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f64f8034000 tx=0x7f64f802e000 comp rx=0 tx=0).stop 2024-04-24T22:40:22.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 -- 172.21.15.47:0/2838004220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650c06bb90 msgr2=0x7f650c1efd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650c06bb90 0x7f650c1efd50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c4a0 tx=0x7f64fc00c960 comp rx=0 tx=0).stop 2024-04-24T22:40:22.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 -- 172.21.15.47:0/2838004220 shutdown_connections 2024-04-24T22:40:22.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f64ec06be30 0x7f64ec06e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f650c073040 0x7f650c1ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f650c072c60 0x7f650c1ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 --2- 172.21.15.47:0/2838004220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f650c06bb90 0x7f650c1efd50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 -- 172.21.15.47:0/2838004220 >> 172.21.15.47:0/2838004220 conn(0x7f650c0cb900 msgr2=0x7f650c13e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.285+0000 7f6510c00640 1 -- 172.21.15.47:0/2838004220 shutdown_connections 2024-04-24T22:40:22.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.286+0000 7f6510c00640 1 -- 172.21.15.47:0/2838004220 wait complete. 2024-04-24T22:40:22.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:40:22.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.524+0000 7f3018c00640 1 Processor -- start 2024-04-24T22:40:22.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.524+0000 7f3018c00640 1 -- start start 2024-04-24T22:40:22.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3018c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3014150de0 0x7f301406b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3018c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30141511c0 0x7f30140b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3018c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30140c2400 0x7f30140c2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3018c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f30140c2230 con 0x7f30141511c0 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3018c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3014073090 con 0x7f3014150de0 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3018c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f301406b7f0 con 0x7f30140c2400 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3011e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3014150de0 0x7f301406b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3011400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30141511c0 0x7f30140b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3012800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30140c2400 0x7f30140c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3011e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3014150de0 0x7f301406b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:37880/0 (socket says 172.21.15.47:37880) 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.525+0000 7f3011e00640 1 -- 172.21.15.47:0/4294583486 learned_addr learned my addr 172.21.15.47:0/4294583486 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.526+0000 7f3011e00640 1 -- 172.21.15.47:0/4294583486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30140c2400 msgr2=0x7f30140c2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.526+0000 7f3011e00640 1 --2- 172.21.15.47:0/4294583486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30140c2400 0x7f30140c2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.526+0000 7f3011e00640 1 -- 172.21.15.47:0/4294583486 >> v2:172.21.15.47:3300/0 conn(0x7f30141511c0 msgr2=0x7f30140b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:22.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.526+0000 7f3011e00640 1 --2- 172.21.15.47:0/4294583486 >> v2:172.21.15.47:3300/0 conn(0x7f30141511c0 0x7f30140b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.526+0000 7f3011e00640 1 -- 172.21.15.47:0/4294583486 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301406bad0 con 0x7f3014150de0 2024-04-24T22:40:22.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.527+0000 7f3011e00640 1 --2- 172.21.15.47:0/4294583486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3014150de0 0x7f301406b160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f301406f540 tx=0x7f2ffc02c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d75b210862425359 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:22.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.527+0000 7f3010a00640 1 -- 172.21.15.47:0/4294583486 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc038070 con 0x7f3014150de0 2024-04-24T22:40:22.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.527+0000 7f3010a00640 1 -- 172.21.15.47:0/4294583486 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc031710 con 0x7f3014150de0 2024-04-24T22:40:22.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.528+0000 7f3010a00640 1 -- 172.21.15.47:0/4294583486 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc031ae0 con 0x7f3014150de0 2024-04-24T22:40:22.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.528+0000 7f3018c00640 1 -- 172.21.15.47:0/4294583486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3014150de0 msgr2=0x7f301406b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.528+0000 7f3018c00640 1 --2- 172.21.15.47:0/4294583486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3014150de0 0x7f301406b160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f301406f540 tx=0x7f2ffc02c3f0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.528+0000 7f3018c00640 1 -- 172.21.15.47:0/4294583486 shutdown_connections 2024-04-24T22:40:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.528+0000 7f3018c00640 1 --2- 172.21.15.47:0/4294583486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f30140c2400 0x7f30140c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.528+0000 7f3018c00640 1 --2- 172.21.15.47:0/4294583486 >> v2:172.21.15.47:3300/0 conn(0x7f30141511c0 0x7f30140b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.528+0000 7f3018c00640 1 --2- 172.21.15.47:0/4294583486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3014150de0 0x7f301406b160 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.528+0000 7f3018c00640 1 -- 172.21.15.47:0/4294583486 >> 172.21.15.47:0/4294583486 conn(0x7f301406e580 msgr2=0x7f301406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.528+0000 7f3018c00640 1 -- 172.21.15.47:0/4294583486 shutdown_connections 2024-04-24T22:40:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.528+0000 7f3018c00640 1 -- 172.21.15.47:0/4294583486 wait complete. 2024-04-24T22:40:22.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.530+0000 7f3018c00640 1 Processor -- start 2024-04-24T22:40:22.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.530+0000 7f3018c00640 1 -- start start 2024-04-24T22:40:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.531+0000 7f3018c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30140c2400 0x7f30141f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.531+0000 7f3018c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3014150de0 0x7f30140c1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.531+0000 7f3018c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3014156130 0x7f30141de320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.531+0000 7f3018c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3014072d20 con 0x7f3014150de0 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.531+0000 7f3018c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3014072a40 con 0x7f30140c2400 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.531+0000 7f3018c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f30140b1e20 con 0x7f3014156130 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.531+0000 7f3011400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3014150de0 0x7f30140c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.532+0000 7f3011400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3014150de0 0x7f30140c1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60604/0 (socket says 172.21.15.47:60604) 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.532+0000 7f3011400640 1 -- 172.21.15.47:0/640850224 learned_addr learned my addr 172.21.15.47:0/640850224 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.532+0000 7f3011e00640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30140c2400 0x7f30141f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.532+0000 7f3012800640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3014156130 0x7f30141de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.532+0000 7f3011400640 1 -- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3014156130 msgr2=0x7f30141de320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.532+0000 7f3011400640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3014156130 0x7f30141de320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.532+0000 7f3011400640 1 -- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30140c2400 msgr2=0x7f30141f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.532+0000 7f3011400640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30140c2400 0x7f30141f0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.532+0000 7f3011400640 1 -- 172.21.15.47:0/640850224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30141565e0 con 0x7f3014150de0 2024-04-24T22:40:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.532+0000 7f3011400640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3014150de0 0x7f30140c1670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3004009d70 tx=0x7f300400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.533+0000 7f300a000640 1 -- 172.21.15.47:0/640850224 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004018020 con 0x7f3014150de0 2024-04-24T22:40:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.533+0000 7f3018c00640 1 -- 172.21.15.47:0/640850224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3014154ab0 con 0x7f3014150de0 2024-04-24T22:40:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.533+0000 7f300a000640 1 -- 172.21.15.47:0/640850224 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3004014480 con 0x7f3014150de0 2024-04-24T22:40:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.533+0000 7f300a000640 1 -- 172.21.15.47:0/640850224 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004014850 con 0x7f3014150de0 2024-04-24T22:40:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.533+0000 7f3018c00640 1 -- 172.21.15.47:0/640850224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3014155070 con 0x7f3014150de0 2024-04-24T22:40:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.534+0000 7f3018c00640 1 -- 172.21.15.47:0/640850224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3014073b90 con 0x7f3014150de0 2024-04-24T22:40:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.535+0000 7f300a000640 1 -- 172.21.15.47:0/640850224 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3004005d10 con 0x7f3014150de0 2024-04-24T22:40:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.536+0000 7f300a000640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ff006be30 0x7f2ff006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.536+0000 7f300a000640 1 -- 172.21.15.47:0/640850224 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f300408bb20 con 0x7f3014150de0 2024-04-24T22:40:22.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.536+0000 7f3011e00640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ff006be30 0x7f2ff006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.537+0000 7f3011e00640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ff006be30 0x7f2ff006e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc035000 tx=0x7f2ffc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.540+0000 7f300a000640 1 -- 172.21.15.47:0/640850224 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3004022030 con 0x7f3014150de0 2024-04-24T22:40:22.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.740+0000 7f3018c00640 1 -- 172.21.15.47:0/640850224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3014150600 con 0x7f3014150de0 2024-04-24T22:40:22.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.741+0000 7f300a000640 1 -- 172.21.15.47:0/640850224 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f3004010040 con 0x7f3014150de0 2024-04-24T22:40:22.743 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.743+0000 7f3018c00640 1 -- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ff006be30 msgr2=0x7f2ff006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.743+0000 7f3018c00640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ff006be30 0x7f2ff006e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc035000 tx=0x7f2ffc008030 comp rx=0 tx=0).stop 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.743+0000 7f3018c00640 1 -- 172.21.15.47:0/640850224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3014150de0 msgr2=0x7f30140c1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.743+0000 7f3018c00640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3014150de0 0x7f30140c1670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3004009d70 tx=0x7f300400c490 comp rx=0 tx=0).stop 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.744+0000 7f3018c00640 1 -- 172.21.15.47:0/640850224 shutdown_connections 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.744+0000 7f3018c00640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2ff006be30 0x7f2ff006e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.744+0000 7f3018c00640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3014156130 0x7f30141de320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.744+0000 7f3018c00640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3014150de0 0x7f30140c1670 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.744+0000 7f3018c00640 1 --2- 172.21.15.47:0/640850224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30140c2400 0x7f30141f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.744+0000 7f3018c00640 1 -- 172.21.15.47:0/640850224 >> 172.21.15.47:0/640850224 conn(0x7f301406e580 msgr2=0x7f30140c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.744+0000 7f3018c00640 1 -- 172.21.15.47:0/640850224 shutdown_connections 2024-04-24T22:40:22.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.744+0000 7f3018c00640 1 -- 172.21.15.47:0/640850224 wait complete. 2024-04-24T22:40:22.763 DEBUG:tasks.mon_thrash:Yielding 2024-04-24T22:40:22.763 INFO:teuthology.run_tasks:Running task workunit... 2024-04-24T22:40:22.781 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:40:22.781 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T22:40:22.781 INFO:tasks.workunit:timeout=3h 2024-04-24T22:40:22.782 INFO:tasks.workunit:cleanup=True 2024-04-24T22:40:22.783 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T22:40:22.785 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:40:22.833 INFO:teuthology.orchestra.run.smithi047.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T22:40:22.833 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-24T22:40:22.833 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 2ah/42d Inode: 1 Links: 2 2024-04-24T22:40:22.833 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T22:40:22.834 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-24T22:40:22.834 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-24T22:40:22.834 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-24 22:39:42.700297138 +0000 2024-04-24T22:40:22.835 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-24 22:40:17.259522351 +0000 2024-04-24T22:40:22.835 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-24 22:39:42.700297138 +0000 2024-04-24T22:40:22.835 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T22:40:22.835 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T22:40:22.952 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:40:22.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.977+0000 7f61c7e00640 1 Processor -- start 2024-04-24T22:40:22.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.977+0000 7f61c7e00640 1 -- start start 2024-04-24T22:40:22.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.978+0000 7f61c7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f61c80c9e50 0x7f61c815dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.978+0000 7f61c7e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c815e290 con 0x7f61c815bae0 2024-04-24T22:40:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.978+0000 7f61c7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c815e570 con 0x7f61c806bf80 2024-04-24T22:40:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.978+0000 7f61c7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f61c815eb30 con 0x7f61c80c9e50 2024-04-24T22:40:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.978+0000 7f61c6000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f61c815bae0 0x7f61c815b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56552/0 (socket says 172.21.15.47:56552) 2024-04-24T22:40:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.979+0000 7f61c6000640 1 -- 172.21.15.47:0/1342365174 learned_addr learned my addr 172.21.15.47:0/1342365174 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.979+0000 7f61c6a00640 1 --2- 172.21.15.47:0/1342365174 >> v2:172.21.15.113:3300/0 conn(0x7f61c80c9e50 0x7f61c815dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.979+0000 7f61c6a00640 1 -- 172.21.15.47:0/1342365174 >> v1:172.21.15.113:6789/0 conn(0x7f61c806bf80 legacy=0x7f61c806b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.979+0000 7f61c6a00640 1 -- 172.21.15.47:0/1342365174 >> v1:172.21.15.47:6789/0 conn(0x7f61c815bae0 legacy=0x7f61c815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.979+0000 7f61c6a00640 1 -- 172.21.15.47:0/1342365174 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c815e850 con 0x7f61c80c9e50 2024-04-24T22:40:22.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.979+0000 7f61c5600640 1 -- 172.21.15.47:0/1342365174 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3216908318 0 0) 0x7f61c815e290 con 0x7f61c815bae0 2024-04-24T22:40:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.980+0000 7f61c6a00640 1 --2- 172.21.15.47:0/1342365174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c80c9e50 0x7f61c815dbe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61b80097a0 tx=0x7f61b802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62e413774be53314 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.980+0000 7f61c5600640 1 -- 172.21.15.47:0/1342365174 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8037070 con 0x7f61c80c9e50 2024-04-24T22:40:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.980+0000 7f61c5600640 1 -- 172.21.15.47:0/1342365174 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b8032640 con 0x7f61c80c9e50 2024-04-24T22:40:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.980+0000 7f61c5600640 1 -- 172.21.15.47:0/1342365174 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8032a10 con 0x7f61c80c9e50 2024-04-24T22:40:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.981+0000 7f61c7e00640 1 -- 172.21.15.47:0/1342365174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c80c9e50 msgr2=0x7f61c815dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.981+0000 7f61c7e00640 1 --2- 172.21.15.47:0/1342365174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c80c9e50 0x7f61c815dbe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61b80097a0 tx=0x7f61b802a2c0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.981+0000 7f61c7e00640 1 -- 172.21.15.47:0/1342365174 shutdown_connections 2024-04-24T22:40:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.981+0000 7f61c7e00640 1 --2- 172.21.15.47:0/1342365174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c80c9e50 0x7f61c815dbe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.981+0000 7f61c7e00640 1 -- 172.21.15.47:0/1342365174 >> 172.21.15.47:0/1342365174 conn(0x7f61c80cfd00 msgr2=0x7f61c80d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.981+0000 7f61c7e00640 1 -- 172.21.15.47:0/1342365174 shutdown_connections 2024-04-24T22:40:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.981+0000 7f61c7e00640 1 -- 172.21.15.47:0/1342365174 wait complete. 2024-04-24T22:40:22.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.983+0000 7f61c7e00640 1 Processor -- start 2024-04-24T22:40:22.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.983+0000 7f61c7e00640 1 -- start start 2024-04-24T22:40:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.984+0000 7f61c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61c806bf80 0x7f61c8154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.984+0000 7f61c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c80c9e50 0x7f61c80cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.984+0000 7f61c7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61c815bae0 0x7f61c80cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.984+0000 7f61c7e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f61c815ab60 con 0x7f61c806bf80 2024-04-24T22:40:22.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.984+0000 7f61c7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61c80cc5d0 con 0x7f61c80c9e50 2024-04-24T22:40:22.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.984+0000 7f61c7e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f61c806b870 con 0x7f61c815bae0 2024-04-24T22:40:22.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.984+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61c806bf80 0x7f61c8154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.984+0000 7f61c7400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61c815bae0 0x7f61c80cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.985+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61c806bf80 0x7f61c8154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60610/0 (socket says 172.21.15.47:60610) 2024-04-24T22:40:22.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.985+0000 7f61c6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c80c9e50 0x7f61c80cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.985+0000 7f61c6a00640 1 -- 172.21.15.47:0/1960719387 learned_addr learned my addr 172.21.15.47:0/1960719387 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.985+0000 7f61c6a00640 1 -- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61c815bae0 msgr2=0x7f61c80cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.985+0000 7f61c6a00640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61c815bae0 0x7f61c80cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.985+0000 7f61c6a00640 1 -- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c80c9e50 msgr2=0x7f61c80cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.985+0000 7f61c6a00640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c80c9e50 0x7f61c80cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.985+0000 7f61c6a00640 1 -- 172.21.15.47:0/1960719387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c8154770 con 0x7f61c806bf80 2024-04-24T22:40:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.985+0000 7f61c7400640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61c815bae0 0x7f61c80cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.985+0000 7f61c6a00640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61c806bf80 0x7f61c8154230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f61b8035040 tx=0x7f61b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.986+0000 7f61bea00640 1 -- 172.21.15.47:0/1960719387 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8041030 con 0x7f61c806bf80 2024-04-24T22:40:22.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.986+0000 7f61bea00640 1 -- 172.21.15.47:0/1960719387 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b8030750 con 0x7f61c806bf80 2024-04-24T22:40:22.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.986+0000 7f61bea00640 1 -- 172.21.15.47:0/1960719387 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8037040 con 0x7f61c806bf80 2024-04-24T22:40:22.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.986+0000 7f61c7e00640 1 -- 172.21.15.47:0/1960719387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c81e43b0 con 0x7f61c806bf80 2024-04-24T22:40:22.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.986+0000 7f61c7e00640 1 -- 172.21.15.47:0/1960719387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c81e4930 con 0x7f61c806bf80 2024-04-24T22:40:22.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.987+0000 7f61c7e00640 1 -- 172.21.15.47:0/1960719387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c80cf550 con 0x7f61c806bf80 2024-04-24T22:40:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.989+0000 7f61bea00640 1 -- 172.21.15.47:0/1960719387 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61b8044070 con 0x7f61c806bf80 2024-04-24T22:40:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.990+0000 7f61bea00640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f61a806be60 0x7f61a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.990+0000 7f61bea00640 1 -- 172.21.15.47:0/1960719387 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f61b80ab660 con 0x7f61c806bf80 2024-04-24T22:40:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.990+0000 7f61c6000640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f61a806be60 0x7f61a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.991+0000 7f61c6000640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f61a806be60 0x7f61a806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61c81f15c0 tx=0x7f61b4007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:22.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:22.994+0000 7f61bea00640 1 -- 172.21.15.47:0/1960719387 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f61c80cf550 con 0x7f61c806bf80 2024-04-24T22:40:22.998 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T22:40:23.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.227+0000 7f61c7e00640 1 -- 172.21.15.47:0/1960719387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61c80cba80 con 0x7f61c806bf80 2024-04-24T22:40:23.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.227+0000 7f61bea00640 1 -- 172.21.15.47:0/1960719387 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6916 (secure 0 0 0) 0x7f61b80799c0 con 0x7f61c806bf80 2024-04-24T22:40:23.229 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:40:23.229 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1712577517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1712577517},{"type":"v1","addr":"172.21.15.113:6839","nonce":1712577517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:39:37.178621+0000","modified":"2024-04-24T22:40:06.837213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4282,"mds_2":4279,"mds_3":4241,"mds_4":4238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4238":{"gid":4238,"name":"a","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6833/2908847255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2908847255},{"type":"v1","addr":"172.21.15.47:6833","nonce":2908847255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.47:6837/2173854280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2173854280},{"type":"v1","addr":"172.21.15.47:6837","nonce":2173854280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/3103107210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3103107210},{"type":"v1","addr":"172.21.15.113:6837","nonce":3103107210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4282":{"gid":4282,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6838/3621458364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3621458364},{"type":"v1","addr":"172.21.15.113:6838","nonce":3621458364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.47:6835/1117134154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1117134154},{"type":"v1","addr":"172.21.15.47:6835","nonce":1117134154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4238,4241,4306,4279,4282]},"id":1}]} 2024-04-24T22:40:23.229 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 -- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f61a806be60 msgr2=0x7f61a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f61a806be60 0x7f61a806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61c81f15c0 tx=0x7f61b4007e40 comp rx=0 tx=0).stop 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 -- 172.21.15.47:0/1960719387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61c806bf80 msgr2=0x7f61c8154230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61c806bf80 0x7f61c8154230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f61b8035040 tx=0x7f61b80046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 -- 172.21.15.47:0/1960719387 shutdown_connections 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f61a806be60 0x7f61a806e320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61c815bae0 0x7f61c80cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c80c9e50 0x7f61c80cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 --2- 172.21.15.47:0/1960719387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61c806bf80 0x7f61c8154230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 -- 172.21.15.47:0/1960719387 >> 172.21.15.47:0/1960719387 conn(0x7f61c80cfd00 msgr2=0x7f61c80b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:23.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.230+0000 7f61c7e00640 1 -- 172.21.15.47:0/1960719387 shutdown_connections 2024-04-24T22:40:23.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.231+0000 7f61c7e00640 1 -- 172.21.15.47:0/1960719387 wait complete. 2024-04-24T22:40:23.252 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-04-24T22:40:23.252 INFO:tasks.mon_thrash.mon_thrasher:seed: 1713998420, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-04-24T22:40:23.252 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-24T22:40:23.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:40:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.504+0000 7f5f2e400640 1 Processor -- start 2024-04-24T22:40:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.504+0000 7f5f2e400640 1 -- start start 2024-04-24T22:40:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.505+0000 7f5f2e400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5f28154660 0x7f5f2806ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.506+0000 7f5f2e400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815e5c0 0x7f5f280c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.506+0000 7f5f2e400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f280c6dc0 con 0x7f5f2806f120 2024-04-24T22:40:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.506+0000 7f5f2e400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5f2815da20 con 0x7f5f28154660 2024-04-24T22:40:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.506+0000 7f5f2e400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5f280c70a0 con 0x7f5f2815e5c0 2024-04-24T22:40:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.506+0000 7f5f27e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5f2806f120 0x7f5f2815deb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56566/0 (socket says 172.21.15.47:56566) 2024-04-24T22:40:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.506+0000 7f5f27e00640 1 -- 172.21.15.47:0/1266722135 learned_addr learned my addr 172.21.15.47:0/1266722135 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.506+0000 7f5f2da00640 1 --2- 172.21.15.47:0/1266722135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815e5c0 0x7f5f280c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.506+0000 7f5f2d000640 1 --2- 172.21.15.47:0/1266722135 >> v2:172.21.15.47:3300/0 conn(0x7f5f28154660 0x7f5f2806ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.507+0000 7f5f27400640 1 -- 172.21.15.47:0/1266722135 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1283270711 0 0) 0x7f5f280c6dc0 con 0x7f5f2806f120 2024-04-24T22:40:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.507+0000 7f5f27400640 1 -- 172.21.15.47:0/1266722135 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f08003520 con 0x7f5f2806f120 2024-04-24T22:40:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.507+0000 7f5f2da00640 1 -- 172.21.15.47:0/1266722135 >> v2:172.21.15.47:3300/0 conn(0x7f5f28154660 msgr2=0x7f5f2806ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.507+0000 7f5f2da00640 1 --2- 172.21.15.47:0/1266722135 >> v2:172.21.15.47:3300/0 conn(0x7f5f28154660 0x7f5f2806ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.507+0000 7f5f2da00640 1 -- 172.21.15.47:0/1266722135 >> v1:172.21.15.47:6789/0 conn(0x7f5f2806f120 legacy=0x7f5f2815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.507+0000 7f5f2da00640 1 -- 172.21.15.47:0/1266722135 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f280c7380 con 0x7f5f2815e5c0 2024-04-24T22:40:23.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.508+0000 7f5f2da00640 1 --2- 172.21.15.47:0/1266722135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815e5c0 0x7f5f280c8eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5f200097a0 tx=0x7f5f2002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be757fa02e48d673 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:23.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.508+0000 7f5f27400640 1 -- 172.21.15.47:0/1266722135 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20037070 con 0x7f5f2815e5c0 2024-04-24T22:40:23.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.508+0000 7f5f27400640 1 -- 172.21.15.47:0/1266722135 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f20032440 con 0x7f5f2815e5c0 2024-04-24T22:40:23.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.508+0000 7f5f27400640 1 -- 172.21.15.47:0/1266722135 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20032810 con 0x7f5f2815e5c0 2024-04-24T22:40:23.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.509+0000 7f5f2e400640 1 -- 172.21.15.47:0/1266722135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815e5c0 msgr2=0x7f5f280c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:23.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.509+0000 7f5f2e400640 1 --2- 172.21.15.47:0/1266722135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815e5c0 0x7f5f280c8eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5f200097a0 tx=0x7f5f2002a210 comp rx=0 tx=0).stop 2024-04-24T22:40:23.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.509+0000 7f5f2e400640 1 -- 172.21.15.47:0/1266722135 shutdown_connections 2024-04-24T22:40:23.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.509+0000 7f5f2e400640 1 --2- 172.21.15.47:0/1266722135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815e5c0 0x7f5f280c8eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.509+0000 7f5f2e400640 1 --2- 172.21.15.47:0/1266722135 >> v2:172.21.15.47:3300/0 conn(0x7f5f28154660 0x7f5f2806ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.509+0000 7f5f2e400640 1 -- 172.21.15.47:0/1266722135 >> 172.21.15.47:0/1266722135 conn(0x7f5f2806cee0 msgr2=0x7f5f2806d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:23.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.509+0000 7f5f2e400640 1 -- 172.21.15.47:0/1266722135 shutdown_connections 2024-04-24T22:40:23.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.509+0000 7f5f2e400640 1 -- 172.21.15.47:0/1266722135 wait complete. 2024-04-24T22:40:23.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.511+0000 7f5f2e400640 1 Processor -- start 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.511+0000 7f5f2e400640 1 -- start start 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f2806f120 0x7f5f280cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2e400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f28154660 0x7f5f2815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815e5c0 0x7f5f281f0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2e400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28154a40 con 0x7f5f2806f120 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280c7940 con 0x7f5f2815e5c0 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2e400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5f28071260 con 0x7f5f28154660 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f2806f120 0x7f5f280cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f2806f120 0x7f5f280cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60626/0 (socket says 172.21.15.47:60626) 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2d000640 1 -- 172.21.15.47:0/842247722 learned_addr learned my addr 172.21.15.47:0/842247722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f27e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f28154660 0x7f5f2815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2da00640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815e5c0 0x7f5f281f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2d000640 1 -- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f28154660 msgr2=0x7f5f2815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2d000640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f28154660 0x7f5f2815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.512+0000 7f5f2d000640 1 -- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815e5c0 msgr2=0x7f5f281f0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.513+0000 7f5f2d000640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815e5c0 0x7f5f281f0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.513+0000 7f5f2d000640 1 -- 172.21.15.47:0/842247722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f280ccc30 con 0x7f5f2806f120 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.513+0000 7f5f2da00640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815e5c0 0x7f5f281f0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.513+0000 7f5f2d000640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f2806f120 0x7f5f280cc580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e9b0 tx=0x7f5f1800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.513+0000 7f5f24c00640 1 -- 172.21.15.47:0/842247722 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18017070 con 0x7f5f2806f120 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.513+0000 7f5f24c00640 1 -- 172.21.15.47:0/842247722 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1800c9a0 con 0x7f5f2806f120 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.513+0000 7f5f24c00640 1 -- 172.21.15.47:0/842247722 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800cd90 con 0x7f5f2806f120 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.513+0000 7f5f2e400640 1 -- 172.21.15.47:0/842247722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f280ccf10 con 0x7f5f2806f120 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.513+0000 7f5f2e400640 1 -- 172.21.15.47:0/842247722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f281f3900 con 0x7f5f2806f120 2024-04-24T22:40:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.514+0000 7f5f2e400640 1 -- 172.21.15.47:0/842247722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2804f5e0 con 0x7f5f2806f120 2024-04-24T22:40:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.516+0000 7f5f24c00640 1 -- 172.21.15.47:0/842247722 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f1801a070 con 0x7f5f2806f120 2024-04-24T22:40:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.516+0000 7f5f24c00640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5f0406be60 0x7f5f0406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.517+0000 7f5f24c00640 1 -- 172.21.15.47:0/842247722 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5f1808b250 con 0x7f5f2806f120 2024-04-24T22:40:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.517+0000 7f5f27e00640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5f0406be60 0x7f5f0406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.518+0000 7f5f27e00640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5f0406be60 0x7f5f0406e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5f2815b450 tx=0x7f5f10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.522+0000 7f5f24c00640 1 -- 172.21.15.47:0/842247722 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5f18059490 con 0x7f5f2806f120 2024-04-24T22:40:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.789+0000 7f5f2e400640 1 -- 172.21.15.47:0/842247722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5f280c9a60 con 0x7f5f2806f120 2024-04-24T22:40:23.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.790+0000 7f5f24c00640 1 -- 172.21.15.47:0/842247722 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f5f1805d790 con 0x7f5f2806f120 2024-04-24T22:40:23.792 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":65,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 -- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5f0406be60 msgr2=0x7f5f0406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5f0406be60 0x7f5f0406e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5f2815b450 tx=0x7f5f10009040 comp rx=0 tx=0).stop 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 -- 172.21.15.47:0/842247722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f2806f120 msgr2=0x7f5f280cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f2806f120 0x7f5f280cc580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e9b0 tx=0x7f5f1800ee70 comp rx=0 tx=0).stop 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 -- 172.21.15.47:0/842247722 shutdown_connections 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5f0406be60 0x7f5f0406e320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815e5c0 0x7f5f281f0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f28154660 0x7f5f2815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 --2- 172.21.15.47:0/842247722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f2806f120 0x7f5f280cc580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 -- 172.21.15.47:0/842247722 >> 172.21.15.47:0/842247722 conn(0x7f5f2806cee0 msgr2=0x7f5f2815cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 -- 172.21.15.47:0/842247722 shutdown_connections 2024-04-24T22:40:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:23.793+0000 7f5f2e400640 1 -- 172.21.15.47:0/842247722 wait complete. 2024-04-24T22:40:23.813 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-24T22:40:23.813 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-24T22:40:23.813 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-24T22:40:23.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:40:24.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.000+0000 7fb614e00640 1 Processor -- start 2024-04-24T22:40:24.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.000+0000 7fb614e00640 1 -- start start 2024-04-24T22:40:24.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb614e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb61006bb90 0x7fb61015b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb614e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb61015c3d0 con 0x7fb610073040 2024-04-24T22:40:24.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb614e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb61015c6b0 con 0x7fb610072c60 2024-04-24T22:40:24.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb614e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb610158bf0 con 0x7fb61006bb90 2024-04-24T22:40:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb60ea00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb610073040 0x7fb6101411f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56580/0 (socket says 172.21.15.47:56580) 2024-04-24T22:40:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb60ea00640 1 -- 172.21.15.47:0/3133278566 learned_addr learned my addr 172.21.15.47:0/3133278566 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb60f400640 1 --1- 172.21.15.47:0/3133278566 >> v1:172.21.15.113:6789/0 conn(0x7fb610072c60 0x7fb6100c2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:59608/0 (socket says 172.21.15.47:59608) 2024-04-24T22:40:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb60fe00640 1 --2- 172.21.15.47:0/3133278566 >> v2:172.21.15.113:3300/0 conn(0x7fb61006bb90 0x7fb61015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb60e000640 1 -- 172.21.15.47:0/3133278566 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3835157098 0 0) 0x7fb61015c6b0 con 0x7fb610072c60 2024-04-24T22:40:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb60e000640 1 -- 172.21.15.47:0/3133278566 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5f4003520 con 0x7fb610072c60 2024-04-24T22:40:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.002+0000 7fb60fe00640 1 -- 172.21.15.47:0/3133278566 >> v1:172.21.15.113:6789/0 conn(0x7fb610072c60 legacy=0x7fb6100c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.003+0000 7fb60e000640 1 -- 172.21.15.47:0/3133278566 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463074445 0 0) 0x7fb61015c3d0 con 0x7fb610073040 2024-04-24T22:40:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.003+0000 7fb60fe00640 1 -- 172.21.15.47:0/3133278566 >> v1:172.21.15.47:6789/0 conn(0x7fb610073040 legacy=0x7fb6101411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.003+0000 7fb60fe00640 1 -- 172.21.15.47:0/3133278566 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6101585a0 con 0x7fb61006bb90 2024-04-24T22:40:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.004+0000 7fb60fe00640 1 --2- 172.21.15.47:0/3133278566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb61006bb90 0x7fb61015b860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb608009d90 tx=0x7fb60802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450089f2b8c044f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.004+0000 7fb60e000640 1 -- 172.21.15.47:0/3133278566 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608007740 con 0x7fb61006bb90 2024-04-24T22:40:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.004+0000 7fb60e000640 1 -- 172.21.15.47:0/3133278566 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6080078e0 con 0x7fb61006bb90 2024-04-24T22:40:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.004+0000 7fb60e000640 1 -- 172.21.15.47:0/3133278566 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608007cb0 con 0x7fb61006bb90 2024-04-24T22:40:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.004+0000 7fb614e00640 1 -- 172.21.15.47:0/3133278566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb61006bb90 msgr2=0x7fb61015b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.004+0000 7fb614e00640 1 --2- 172.21.15.47:0/3133278566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb61006bb90 0x7fb61015b860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb608009d90 tx=0x7fb60802a6f0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.004+0000 7fb614e00640 1 -- 172.21.15.47:0/3133278566 shutdown_connections 2024-04-24T22:40:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.004+0000 7fb614e00640 1 --2- 172.21.15.47:0/3133278566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb61006bb90 0x7fb61015b860 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.004+0000 7fb614e00640 1 -- 172.21.15.47:0/3133278566 >> 172.21.15.47:0/3133278566 conn(0x7fb6100cb900 msgr2=0x7fb6100cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.005+0000 7fb614e00640 1 -- 172.21.15.47:0/3133278566 shutdown_connections 2024-04-24T22:40:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.005+0000 7fb614e00640 1 -- 172.21.15.47:0/3133278566 wait complete. 2024-04-24T22:40:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.006+0000 7fb614e00640 1 Processor -- start 2024-04-24T22:40:24.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.006+0000 7fb614e00640 1 -- start start 2024-04-24T22:40:24.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.007+0000 7fb614e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb61006bb90 0x7fb6100c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.007+0000 7fb614e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb610072c60 0x7fb610154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.007+0000 7fb614e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb610073040 0x7fb610153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.007+0000 7fb614e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb610159080 con 0x7fb61006bb90 2024-04-24T22:40:24.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.007+0000 7fb614e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb610158270 con 0x7fb610072c60 2024-04-24T22:40:24.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.007+0000 7fb614e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb610159200 con 0x7fb610073040 2024-04-24T22:40:24.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb61006bb90 0x7fb6100c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb610072c60 0x7fb610154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb61006bb90 0x7fb6100c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60630/0 (socket says 172.21.15.47:60630) 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60f400640 1 -- 172.21.15.47:0/2265899877 learned_addr learned my addr 172.21.15.47:0/2265899877 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60fe00640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb610073040 0x7fb610153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60ea00640 1 -- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb610073040 msgr2=0x7fb610153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60ea00640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb610073040 0x7fb610153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60ea00640 1 -- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb61006bb90 msgr2=0x7fb6100c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60ea00640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb61006bb90 0x7fb6100c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60ea00640 1 -- 172.21.15.47:0/2265899877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6101546d0 con 0x7fb610072c60 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.008+0000 7fb60ea00640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb610072c60 0x7fb610154940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00c4b0 tx=0x7fb5fc00c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.009+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc018070 con 0x7fb610072c60 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.009+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5fc0074e0 con 0x7fb610072c60 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.009+0000 7fb614e00640 1 -- 172.21.15.47:0/2265899877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb610154f80 con 0x7fb610072c60 2024-04-24T22:40:24.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.009+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc004a20 con 0x7fb610072c60 2024-04-24T22:40:24.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.009+0000 7fb614e00640 1 -- 172.21.15.47:0/2265899877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb610152ba0 con 0x7fb610072c60 2024-04-24T22:40:24.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.010+0000 7fb614e00640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb610152da0 0x7fb6101f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.010+0000 7fb614e00640 1 -- 172.21.15.47:0/2265899877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb610155c40 con 0x7fb610152da0 2024-04-24T22:40:24.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.010+0000 7fb60f400640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb610152da0 0x7fb6101f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.010+0000 7fb60f400640 1 -- 172.21.15.47:0/2265899877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb61015c3d0 con 0x7fb610152da0 2024-04-24T22:40:24.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.011+0000 7fb60f400640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb610152da0 0x7fb6101f0b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb600000f60 tx=0x7fb600001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.011+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5fc01b070 con 0x7fb610072c60 2024-04-24T22:40:24.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.012+0000 7fb607400640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb5f006be30 0x7fb5f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.012+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb5fc08a1a0 con 0x7fb610072c60 2024-04-24T22:40:24.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.012+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60000f7d0 con 0x7fb610152da0 2024-04-24T22:40:24.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.012+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb600018070 con 0x7fb610152da0 2024-04-24T22:40:24.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.012+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb610152da0 msgr2=0x7fb6101f0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.012+0000 7fb607400640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb610152da0 0x7fb6101f0b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb600000f60 tx=0x7fb600001a50 comp rx=0 tx=0).stop 2024-04-24T22:40:24.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.012+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb610152da0 msgr2=0x7fb6101f0b80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:40:24.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.012+0000 7fb607400640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb610152da0 0x7fb6101f0b80 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb600000f60 tx=0x7fb600001a50 comp rx=0 tx=0).stop 2024-04-24T22:40:24.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.012+0000 7fb60fe00640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb5f006be30 0x7fb5f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.013+0000 7fb60fe00640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb5f006be30 0x7fb5f006e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb608034000 tx=0x7fb60802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.019+0000 7fb614e00640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6101596a0 0x7fb61013df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.019+0000 7fb614e00640 1 -- 172.21.15.47:0/2265899877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb610152a00 con 0x7fb6101596a0 2024-04-24T22:40:24.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.019+0000 7fb60f400640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6101596a0 0x7fb61013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.020+0000 7fb60f400640 1 -- 172.21.15.47:0/2265899877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb610140920 con 0x7fb6101596a0 2024-04-24T22:40:24.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.020+0000 7fb60f400640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6101596a0 0x7fb61013df30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb600000f60 tx=0x7fb600001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.020+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600027410 con 0x7fb6101596a0 2024-04-24T22:40:24.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4726 (secure 0 0 0) 0x7fb6000133f0 con 0x7fb6101596a0 2024-04-24T22:40:24.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6101596a0 msgr2=0x7fb61013df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb607400640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6101596a0 0x7fb61013df30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb600000f60 tx=0x7fb600001150 comp rx=0 tx=0).stop 2024-04-24T22:40:24.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb607400640 1 -- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6101596a0 msgr2=0x7fb61013df30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:40:24.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb607400640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6101596a0 0x7fb61013df30 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb600000f60 tx=0x7fb600001150 comp rx=0 tx=0).stop 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 4, 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 65523, 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:40:24.023 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 65, 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:24.024 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:24.025 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:40:24.026 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:24.027 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:40:24.029 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:40:24.030 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:24.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:24.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:24.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:40:24.030 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.032 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.033 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:24.034 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.035 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.035 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:40:24.035 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:40:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb614e00640 1 -- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb5f006be30 msgr2=0x7fb5f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb614e00640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb5f006be30 0x7fb5f006e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb608034000 tx=0x7fb60802f000 comp rx=0 tx=0).stop 2024-04-24T22:40:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb614e00640 1 -- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb610072c60 msgr2=0x7fb610154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb614e00640 1 --2- 172.21.15.47:0/2265899877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb610072c60 0x7fb610154940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00c4b0 tx=0x7fb5fc00c970 comp rx=0 tx=0).stop 2024-04-24T22:40:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb60fe00640 1 -- 172.21.15.47:0/2265899877 reap_dead start 2024-04-24T22:40:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb60fe00640 1 -- 172.21.15.47:0/2265899877 reap_dead start 2024-04-24T22:40:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb614e00640 1 -- 172.21.15.47:0/2265899877 shutdown_connections 2024-04-24T22:40:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb614e00640 1 -- 172.21.15.47:0/2265899877 >> 172.21.15.47:0/2265899877 conn(0x7fb6100cb900 msgr2=0x7fb6100cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb614e00640 1 -- 172.21.15.47:0/2265899877 shutdown_connections 2024-04-24T22:40:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.021+0000 7fb614e00640 1 -- 172.21.15.47:0/2265899877 wait complete. 2024-04-24T22:40:24.039 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:40:24.039 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-24T22:40:24.039 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:40:24.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.223+0000 7f4993e00640 1 Processor -- start 2024-04-24T22:40:24.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.223+0000 7f4993e00640 1 -- start start 2024-04-24T22:40:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.225+0000 7f4993e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49940cbb40 0x7f49940cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.225+0000 7f4993e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4994073280 0x7f49940ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.225+0000 7f4993e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499415b2c0 con 0x7f4994154560 2024-04-24T22:40:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.225+0000 7f4993e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f499415da90 con 0x7f4994073280 2024-04-24T22:40:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.225+0000 7f4993e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f499415dd70 con 0x7f49940cbb40 2024-04-24T22:40:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.226+0000 7f4993400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4994073280 0x7f49940ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.226+0000 7f4993400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4994073280 0x7f49940ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60668/0 (socket says 172.21.15.47:60668) 2024-04-24T22:40:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.226+0000 7f4992000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49940cbb40 0x7f49940cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.226+0000 7f4993400640 1 -- 172.21.15.47:0/3440910101 learned_addr learned my addr 172.21.15.47:0/3440910101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:24.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.226+0000 7f4992000640 1 -- 172.21.15.47:0/3440910101 >> v2:172.21.15.47:3300/0 conn(0x7f4994073280 msgr2=0x7f49940ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:24.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.226+0000 7f4992000640 1 --2- 172.21.15.47:0/3440910101 >> v2:172.21.15.47:3300/0 conn(0x7f4994073280 0x7f49940ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.226+0000 7f4992000640 1 -- 172.21.15.47:0/3440910101 >> v1:172.21.15.113:6789/0 conn(0x7f4994154560 legacy=0x7f49940cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.226+0000 7f4991600640 1 -- 172.21.15.47:0/3440910101 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2151031948 0 0) 0x7f499415b2c0 con 0x7f4994154560 2024-04-24T22:40:24.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.226+0000 7f4992000640 1 -- 172.21.15.47:0/3440910101 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499415e050 con 0x7f49940cbb40 2024-04-24T22:40:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.227+0000 7f4992000640 1 --2- 172.21.15.47:0/3440910101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49940cbb40 0x7f49940cc660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f499406c5d0 tx=0x7f497c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc5e8a5d5425ac88 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4991600640 1 -- 172.21.15.47:0/3440910101 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c007740 con 0x7f49940cbb40 2024-04-24T22:40:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4991600640 1 -- 172.21.15.47:0/3440910101 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c0078e0 con 0x7f49940cbb40 2024-04-24T22:40:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4991600640 1 -- 172.21.15.47:0/3440910101 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c007cb0 con 0x7f49940cbb40 2024-04-24T22:40:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4993e00640 1 -- 172.21.15.47:0/3440910101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49940cbb40 msgr2=0x7f49940cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4993e00640 1 --2- 172.21.15.47:0/3440910101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49940cbb40 0x7f49940cc660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f499406c5d0 tx=0x7f497c02a5f0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4993e00640 1 -- 172.21.15.47:0/3440910101 shutdown_connections 2024-04-24T22:40:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4993e00640 1 --2- 172.21.15.47:0/3440910101 >> v2:172.21.15.47:3300/0 conn(0x7f4994073280 0x7f49940ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4993e00640 1 --2- 172.21.15.47:0/3440910101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49940cbb40 0x7f49940cc660 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4993e00640 1 -- 172.21.15.47:0/3440910101 >> 172.21.15.47:0/3440910101 conn(0x7f49940672c0 msgr2=0x7f4994072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4993e00640 1 -- 172.21.15.47:0/3440910101 shutdown_connections 2024-04-24T22:40:24.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.228+0000 7f4993e00640 1 -- 172.21.15.47:0/3440910101 wait complete. 2024-04-24T22:40:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.230+0000 7f4993e00640 1 Processor -- start 2024-04-24T22:40:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.230+0000 7f4993e00640 1 -- start start 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4993e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994073280 0x7f4994151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4993e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49940cbb40 0x7f49940cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4993e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994154560 0x7f49940d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4993e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f499415ae80 con 0x7f4994154560 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4993e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49940caad0 con 0x7f4994073280 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4993e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f499415b000 con 0x7f49940cbb40 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4993400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994154560 0x7f49940d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4993400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994154560 0x7f49940d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60670/0 (socket says 172.21.15.47:60670) 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4993400640 1 -- 172.21.15.47:0/785390176 learned_addr learned my addr 172.21.15.47:0/785390176 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4992000640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49940cbb40 0x7f49940cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.231+0000 7f4992a00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994073280 0x7f4994151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f4993400640 1 -- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49940cbb40 msgr2=0x7f49940cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f4993400640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49940cbb40 0x7f49940cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f4993400640 1 -- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994073280 msgr2=0x7f4994151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f4993400640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994073280 0x7f4994151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f4993400640 1 -- 172.21.15.47:0/785390176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49941f4540 con 0x7f4994154560 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f4993400640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994154560 0x7f49940d01b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f498c00e8a0 tx=0x7f498c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c017070 con 0x7f4994154560 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f498c00c940 con 0x7f4994154560 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c00cd50 con 0x7f4994154560 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f4993e00640 1 -- 172.21.15.47:0/785390176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49941f07d0 con 0x7f4994154560 2024-04-24T22:40:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.232+0000 7f4993e00640 1 -- 172.21.15.47:0/785390176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49941f0a70 con 0x7f4994154560 2024-04-24T22:40:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.233+0000 7f4993e00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994157c80 0x7f499415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.233+0000 7f4993e00640 1 -- 172.21.15.47:0/785390176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49941f4aa0 con 0x7f4994157c80 2024-04-24T22:40:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.234+0000 7f4992a00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994157c80 0x7f499415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.234+0000 7f4992a00640 1 -- 172.21.15.47:0/785390176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4994152850 con 0x7f4994157c80 2024-04-24T22:40:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.234+0000 7f4992a00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994157c80 0x7f499415c610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f498400f540 tx=0x7f498400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.235+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f498c01a070 con 0x7f4994154560 2024-04-24T22:40:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.235+0000 7f498aa00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497006be60 0x7f497006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.236+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49840052e0 con 0x7f4994157c80 2024-04-24T22:40:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.236+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f498c08ad30 con 0x7f4994154560 2024-04-24T22:40:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.236+0000 7f4992000640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497006be60 0x7f497006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.236+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4984018070 con 0x7f4994157c80 2024-04-24T22:40:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.236+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994157c80 msgr2=0x7f499415c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.236+0000 7f498aa00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994157c80 0x7f499415c610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f498400f540 tx=0x7f498400fa00 comp rx=0 tx=0).stop 2024-04-24T22:40:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.236+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994157c80 msgr2=0x7f499415c610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:40:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.236+0000 7f498aa00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4994157c80 0x7f499415c610 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f498400f540 tx=0x7f498400fa00 comp rx=0 tx=0).stop 2024-04-24T22:40:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.236+0000 7f4992000640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497006be60 0x7f497006e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f497c034000 tx=0x7f497c02f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.243+0000 7f4993e00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499415cb50 0x7f499415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.243+0000 7f4993e00640 1 -- 172.21.15.47:0/785390176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49941f1530 con 0x7f499415cb50 2024-04-24T22:40:24.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.243+0000 7f4992a00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499415cb50 0x7f499415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.244+0000 7f4992a00640 1 -- 172.21.15.47:0/785390176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f499415d470 con 0x7f499415cb50 2024-04-24T22:40:24.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.244+0000 7f4992a00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499415cb50 0x7f499415cf30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f498400f540 tx=0x7f4984001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.244+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49840038c0 con 0x7f499415cb50 2024-04-24T22:40:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5064 (secure 0 0 0) 0x7f49840038c0 con 0x7f499415cb50 2024-04-24T22:40:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499415cb50 msgr2=0x7f499415cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f498aa00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499415cb50 0x7f499415cf30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f498400f540 tx=0x7f4984001080 comp rx=0 tx=0).stop 2024-04-24T22:40:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f498aa00640 1 -- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499415cb50 msgr2=0x7f499415cf30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:40:24.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f498aa00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499415cb50 0x7f499415cf30 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f498400f540 tx=0x7f4984001080 comp rx=0 tx=0).stop 2024-04-24T22:40:24.247 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:40:24.247 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:40:24.247 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:40:24.247 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:40:24.247 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 4, 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 65812, 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 65, 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:40:24.248 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:24.249 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.250 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:40:24.250 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:40:24.250 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:24.250 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:24.250 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:24.250 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:24.250 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:24.250 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:24.250 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:24.250 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:40:24.251 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:24.252 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.253 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:24.254 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:40:24.255 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.255 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.255 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.255 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:40:24.255 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:40:24.255 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:24.255 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:24.255 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:24.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:24.256 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:24.257 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:40:24.258 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.259 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:40:24.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:40:24.261 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:40:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f4993e00640 1 -- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497006be60 msgr2=0x7f497006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f4993e00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f497006be60 0x7f497006e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f497c034000 tx=0x7f497c02f000 comp rx=0 tx=0).stop 2024-04-24T22:40:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f4993e00640 1 -- 172.21.15.47:0/785390176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994154560 msgr2=0x7f49940d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f4993e00640 1 --2- 172.21.15.47:0/785390176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4994154560 0x7f49940d01b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f498c00e8a0 tx=0x7f498c00ed60 comp rx=0 tx=0).stop 2024-04-24T22:40:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f4993400640 1 -- 172.21.15.47:0/785390176 reap_dead start 2024-04-24T22:40:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f4993400640 1 -- 172.21.15.47:0/785390176 reap_dead start 2024-04-24T22:40:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f4993e00640 1 -- 172.21.15.47:0/785390176 shutdown_connections 2024-04-24T22:40:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f4993e00640 1 -- 172.21.15.47:0/785390176 >> 172.21.15.47:0/785390176 conn(0x7f49940672c0 msgr2=0x7f499414a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.245+0000 7f4993e00640 1 -- 172.21.15.47:0/785390176 shutdown_connections 2024-04-24T22:40:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.246+0000 7f4993e00640 1 -- 172.21.15.47:0/785390176 wait complete. 2024-04-24T22:40:24.264 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:40:24.264 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-24T22:40:24.264 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-24T22:40:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.472+0000 7f9ca3e00640 1 Processor -- start 2024-04-24T22:40:24.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.473+0000 7f9ca3e00640 1 -- start start 2024-04-24T22:40:24.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.474+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 0x7f9ca40ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.474+0000 7f9ca3e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca415b2c0 con 0x7f9ca4154560 2024-04-24T22:40:24.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.474+0000 7f9ca3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca415da90 con 0x7f9ca40cbb40 2024-04-24T22:40:24.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.474+0000 7f9ca3e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca415dd70 con 0x7f9ca4073280 2024-04-24T22:40:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.475+0000 7f9ca3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 0x7f9ca40ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.475+0000 7f9ca2a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9ca4154560 0x7f9ca40cb430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56594/0 (socket says 172.21.15.47:56594) 2024-04-24T22:40:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.475+0000 7f9ca2a00640 1 -- 172.21.15.47:0/3889067640 learned_addr learned my addr 172.21.15.47:0/3889067640 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.475+0000 7f9ca2000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9ca40cbb40 0x7f9ca40cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:59634/0 (socket says 172.21.15.47:59634) 2024-04-24T22:40:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.475+0000 7f9ca1600640 1 -- 172.21.15.47:0/3889067640 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352670775 0 0) 0x7f9ca415da90 con 0x7f9ca40cbb40 2024-04-24T22:40:24.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.475+0000 7f9ca3400640 1 -- 172.21.15.47:0/3889067640 >> v1:172.21.15.113:6789/0 conn(0x7f9ca40cbb40 legacy=0x7f9ca40cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.475+0000 7f9ca3400640 1 -- 172.21.15.47:0/3889067640 >> v1:172.21.15.47:6789/0 conn(0x7f9ca4154560 legacy=0x7f9ca40cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.475+0000 7f9ca3400640 1 -- 172.21.15.47:0/3889067640 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca415e050 con 0x7f9ca4073280 2024-04-24T22:40:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.476+0000 7f9ca3400640 1 --2- 172.21.15.47:0/3889067640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 0x7f9ca40ccd70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0097d0 tx=0x7f9c9c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d10f1997a8795339 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.477+0000 7f9ca1600640 1 -- 172.21.15.47:0/3889067640 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c037070 con 0x7f9ca4073280 2024-04-24T22:40:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.477+0000 7f9ca1600640 1 -- 172.21.15.47:0/3889067640 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9c032440 con 0x7f9ca4073280 2024-04-24T22:40:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.477+0000 7f9ca1600640 1 -- 172.21.15.47:0/3889067640 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c032810 con 0x7f9ca4073280 2024-04-24T22:40:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.477+0000 7f9ca3e00640 1 -- 172.21.15.47:0/3889067640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 msgr2=0x7f9ca40ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.477+0000 7f9ca3e00640 1 --2- 172.21.15.47:0/3889067640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 0x7f9ca40ccd70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0097d0 tx=0x7f9c9c02a210 comp rx=0 tx=0).stop 2024-04-24T22:40:24.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.477+0000 7f9ca3e00640 1 -- 172.21.15.47:0/3889067640 shutdown_connections 2024-04-24T22:40:24.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.477+0000 7f9ca3e00640 1 --2- 172.21.15.47:0/3889067640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 0x7f9ca40ccd70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.477+0000 7f9ca3e00640 1 -- 172.21.15.47:0/3889067640 >> 172.21.15.47:0/3889067640 conn(0x7f9ca40672c0 msgr2=0x7f9ca4072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:24.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.478+0000 7f9ca3e00640 1 -- 172.21.15.47:0/3889067640 shutdown_connections 2024-04-24T22:40:24.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.478+0000 7f9ca3e00640 1 -- 172.21.15.47:0/3889067640 wait complete. 2024-04-24T22:40:24.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.479+0000 7f9ca3e00640 1 Processor -- start 2024-04-24T22:40:24.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.479+0000 7f9ca3e00640 1 -- start start 2024-04-24T22:40:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.480+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 0x7f9ca4159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.480+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40cbb40 0x7f9ca415c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.480+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca4154560 0x7f9ca40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.480+0000 7f9ca3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca40cd2b0 con 0x7f9ca40cbb40 2024-04-24T22:40:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.480+0000 7f9ca3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca40caad0 con 0x7f9ca4154560 2024-04-24T22:40:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.480+0000 7f9ca3e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca415ae80 con 0x7f9ca4073280 2024-04-24T22:40:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40cbb40 0x7f9ca415c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca2a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 0x7f9ca4159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40cbb40 0x7f9ca415c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60682/0 (socket says 172.21.15.47:60682) 2024-04-24T22:40:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca2000640 1 -- 172.21.15.47:0/1161349942 learned_addr learned my addr 172.21.15.47:0/1161349942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:24.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca3400640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca4154560 0x7f9ca40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca2a00640 1 -- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca4154560 msgr2=0x7f9ca40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca2a00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca4154560 0x7f9ca40ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca2a00640 1 -- 172.21.15.47:0/1161349942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40cbb40 msgr2=0x7f9ca415c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca2a00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca40cbb40 0x7f9ca415c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca2a00640 1 -- 172.21.15.47:0/1161349942 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca4159bc0 con 0x7f9ca4073280 2024-04-24T22:40:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.481+0000 7f9ca2a00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 0x7f9ca4159430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400c470 tx=0x7f9c94001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.482+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94017070 con 0x7f9ca4073280 2024-04-24T22:40:24.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.482+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c94005ce0 con 0x7f9ca4073280 2024-04-24T22:40:24.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.482+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c940072d0 con 0x7f9ca4073280 2024-04-24T22:40:24.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.482+0000 7f9ca3e00640 1 -- 172.21.15.47:0/1161349942 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca41583a0 con 0x7f9ca4073280 2024-04-24T22:40:24.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.482+0000 7f9ca3e00640 1 -- 172.21.15.47:0/1161349942 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca4158960 con 0x7f9ca4073280 2024-04-24T22:40:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.483+0000 7f9ca3e00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca40d06f0 0x7f9ca40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.483+0000 7f9ca3e00640 1 -- 172.21.15.47:0/1161349942 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca4159ea0 con 0x7f9ca40d06f0 2024-04-24T22:40:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.483+0000 7f9ca2000640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca40d06f0 0x7f9ca40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.484+0000 7f9ca2000640 1 -- 172.21.15.47:0/1161349942 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ca4159970 con 0x7f9ca40d06f0 2024-04-24T22:40:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.484+0000 7f9ca2000640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca40d06f0 0x7f9ca40ceaa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0012e0 tx=0x7f9c8c0079d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.484+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00fa70 con 0x7f9ca40d06f0 2024-04-24T22:40:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.484+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c9401a070 con 0x7f9ca4073280 2024-04-24T22:40:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.485+0000 7f9c9aa00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9c8006bd60 0x7f9c8006e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.485+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9c9408a0c0 con 0x7f9ca4073280 2024-04-24T22:40:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.485+0000 7f9ca3400640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9c8006bd60 0x7f9c8006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.485+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9c8c018070 con 0x7f9ca40d06f0 2024-04-24T22:40:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.485+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca40d06f0 msgr2=0x7f9ca40ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.485+0000 7f9c9aa00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca40d06f0 0x7f9ca40ceaa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0012e0 tx=0x7f9c8c0079d0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.485+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca40d06f0 msgr2=0x7f9ca40ceaa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:40:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.485+0000 7f9c9aa00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca40d06f0 0x7f9ca40ceaa0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0012e0 tx=0x7f9c8c0079d0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.486+0000 7f9ca3400640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9c8006bd60 0x7f9c8006e220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c034000 tx=0x7f9c9c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.493+0000 7f9ca3e00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca414d6d0 0x7f9ca414dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.493+0000 7f9ca3e00640 1 -- 172.21.15.47:0/1161349942 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca4151b50 con 0x7f9ca414d6d0 2024-04-24T22:40:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.493+0000 7f9ca2000640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca414d6d0 0x7f9ca414dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.494+0000 7f9ca2000640 1 -- 172.21.15.47:0/1161349942 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9ca4152f50 con 0x7f9ca414d6d0 2024-04-24T22:40:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.494+0000 7f9ca2000640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca414d6d0 0x7f9ca414dae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c007b60 tx=0x7f9c8c001250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.494+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c0136c0 con 0x7f9ca414d6d0 2024-04-24T22:40:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.494+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5207 (secure 0 0 0) 0x7f9c8c0136c0 con 0x7f9ca414d6d0 2024-04-24T22:40:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca414d6d0 msgr2=0x7f9ca414dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9c9aa00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca414d6d0 0x7f9ca414dae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c007b60 tx=0x7f9c8c001250 comp rx=0 tx=0).stop 2024-04-24T22:40:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9c9aa00640 1 -- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca414d6d0 msgr2=0x7f9ca414dae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:40:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9c9aa00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca414d6d0 0x7f9ca414dae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 4, 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 66061, 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.497 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 65, 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:24.498 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:24.499 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.499 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:40:24.499 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:40:24.499 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:24.499 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:24.499 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:24.499 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:24.499 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:40:24.500 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:24.501 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:40:24.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:40:24.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.505 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.506 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:24.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:40:24.507 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:24.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:40:24.507 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:24.507 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.507 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:40:24.508 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:40:24.508 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:24.508 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:24.508 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:24.508 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.508 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.508 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:40:24.509 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:40:24.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.509 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.509 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.509 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:24.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.509 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.509 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:40:24.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.509 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:40:24.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.511 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:40:24.511 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.511 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:24.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.511 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.511 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.511 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-24T22:40:24.511 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.511 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:40:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:24.512 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:40:24.512 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:40:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9ca3e00640 1 -- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9c8006bd60 msgr2=0x7f9c8006e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9ca3e00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f9c8006bd60 0x7f9c8006e220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c034000 tx=0x7f9c9c02e000 comp rx=0 tx=0).stop 2024-04-24T22:40:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9ca3e00640 1 -- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 msgr2=0x7f9ca4159430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9ca3e00640 1 --2- 172.21.15.47:0/1161349942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4073280 0x7f9ca4159430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400c470 tx=0x7f9c94001e70 comp rx=0 tx=0).stop 2024-04-24T22:40:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9ca3400640 1 -- 172.21.15.47:0/1161349942 reap_dead start 2024-04-24T22:40:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9ca3400640 1 -- 172.21.15.47:0/1161349942 reap_dead start 2024-04-24T22:40:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9ca3e00640 1 -- 172.21.15.47:0/1161349942 shutdown_connections 2024-04-24T22:40:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9ca3e00640 1 -- 172.21.15.47:0/1161349942 >> 172.21.15.47:0/1161349942 conn(0x7f9ca40672c0 msgr2=0x7f9ca40cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9ca3e00640 1 -- 172.21.15.47:0/1161349942 shutdown_connections 2024-04-24T22:40:24.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.495+0000 7f9ca3e00640 1 -- 172.21.15.47:0/1161349942 wait complete. 2024-04-24T22:40:24.516 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:40:24.516 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-04-24T22:40:24.516 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-24T22:40:24.516 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-04-24T22:40:24.516 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-24T22:40:24.517 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-24T22:40:24.517 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:40:24.542 INFO:tasks.ceph.mon.a:Stopped 2024-04-24T22:40:24.543 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-24T22:40:24.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:40:24.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.805+0000 7f3ce2a00640 1 Processor -- start 2024-04-24T22:40:24.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.805+0000 7f3ce2a00640 1 -- start start 2024-04-24T22:40:24.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.807+0000 7f3ce2a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3cdc158be0 0x7f3cdc0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.807+0000 7f3ce2a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc158fc0 0x7f3cdc0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.807+0000 7f3ce2a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cdc1621d0 con 0x7f3cdc0c1280 2024-04-24T22:40:24.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.807+0000 7f3ce2a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3cdc1624b0 con 0x7f3cdc158be0 2024-04-24T22:40:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.807+0000 7f3ce2a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3cdc162790 con 0x7f3cdc158fc0 2024-04-24T22:40:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.807+0000 7f3cdbe00640 1 -- >> v2:172.21.15.47:3300/0 conn(0x7f3cdc158be0 msgr2=0x7f3cdc0c78f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.807+0000 7f3cdb400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc158fc0 0x7f3cdc0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.807+0000 7f3cdbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3cdc158be0 0x7f3cdc0c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.807+0000 7f3ce0a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3cdc0c1280 0x7f3cdc15ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:59642/0 (socket says 172.21.15.47:59642) 2024-04-24T22:40:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.808+0000 7f3ce0a00640 1 -- 172.21.15.47:0/3143453472 learned_addr learned my addr 172.21.15.47:0/3143453472 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.808+0000 7f3cdb400640 1 --2- 172.21.15.47:0/3143453472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc158fc0 0x7f3cdc0c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34524/0 (socket says 172.21.15.47:34524) 2024-04-24T22:40:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.808+0000 7f3cdb400640 1 -- 172.21.15.47:0/3143453472 >> v2:172.21.15.47:3300/0 conn(0x7f3cdc158be0 msgr2=0x7f3cdc0c78f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:40:24.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.808+0000 7f3cdb400640 1 --2- 172.21.15.47:0/3143453472 >> v2:172.21.15.47:3300/0 conn(0x7f3cdc158be0 0x7f3cdc0c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.808+0000 7f3cdaa00640 1 -- 172.21.15.47:0/3143453472 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1474402020 0 0) 0x7f3cdc1621d0 con 0x7f3cdc0c1280 2024-04-24T22:40:24.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.808+0000 7f3cdb400640 1 -- 172.21.15.47:0/3143453472 >> v1:172.21.15.113:6789/0 conn(0x7f3cdc0c1280 legacy=0x7f3cdc15ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.808+0000 7f3cdb400640 1 -- 172.21.15.47:0/3143453472 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc162a70 con 0x7f3cdc158fc0 2024-04-24T22:40:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.809+0000 7f3cdb400640 1 --2- 172.21.15.47:0/3143453472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc158fc0 0x7f3cdc0c8590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80097a0 tx=0x7f3cc802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f48333b9d17fc17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.809+0000 7f3cdaa00640 1 -- 172.21.15.47:0/3143453472 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8037070 con 0x7f3cdc158fc0 2024-04-24T22:40:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.809+0000 7f3cdaa00640 1 -- 172.21.15.47:0/3143453472 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc8032440 con 0x7f3cdc158fc0 2024-04-24T22:40:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.809+0000 7f3cdaa00640 1 -- 172.21.15.47:0/3143453472 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8032810 con 0x7f3cdc158fc0 2024-04-24T22:40:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.810+0000 7f3ce2a00640 1 -- 172.21.15.47:0/3143453472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc158fc0 msgr2=0x7f3cdc0c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.810+0000 7f3ce2a00640 1 --2- 172.21.15.47:0/3143453472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc158fc0 0x7f3cdc0c8590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80097a0 tx=0x7f3cc802a230 comp rx=0 tx=0).stop 2024-04-24T22:40:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.810+0000 7f3ce2a00640 1 -- 172.21.15.47:0/3143453472 shutdown_connections 2024-04-24T22:40:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.810+0000 7f3ce2a00640 1 --2- 172.21.15.47:0/3143453472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc158fc0 0x7f3cdc0c8590 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.810+0000 7f3ce2a00640 1 --2- 172.21.15.47:0/3143453472 >> v2:172.21.15.47:3300/0 conn(0x7f3cdc158be0 0x7f3cdc0c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.810+0000 7f3ce2a00640 1 -- 172.21.15.47:0/3143453472 >> 172.21.15.47:0/3143453472 conn(0x7f3cdc06e580 msgr2=0x7f3cdc06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.810+0000 7f3ce2a00640 1 -- 172.21.15.47:0/3143453472 shutdown_connections 2024-04-24T22:40:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.810+0000 7f3ce2a00640 1 -- 172.21.15.47:0/3143453472 wait complete. 2024-04-24T22:40:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.812+0000 7f3ce2a00640 1 Processor -- start 2024-04-24T22:40:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.812+0000 7f3ce2a00640 1 -- start start 2024-04-24T22:40:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3ce2a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc0c1280 0x7f3cdc160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3ce2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cdc158be0 0x7f3cdc14d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3ce2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cdc158fc0 0x7f3cdc1f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3ce2a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc0c1050 con 0x7f3cdc158fc0 2024-04-24T22:40:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3ce2a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc0c0ed0 con 0x7f3cdc158be0 2024-04-24T22:40:24.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3ce2a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3cdc15f2e0 con 0x7f3cdc0c1280 2024-04-24T22:40:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3cdbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc0c1280 0x7f3cdc160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3cdb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cdc158be0 0x7f3cdc14d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3ce0a00640 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cdc158fc0 msgr2=0x7f3cdc1f77c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3cdbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc0c1280 0x7f3cdc160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34536/0 (socket says 172.21.15.47:34536) 2024-04-24T22:40:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3ce0a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cdc158fc0 0x7f3cdc1f77c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3cdbe00640 1 -- 172.21.15.47:0/1419149819 learned_addr learned my addr 172.21.15.47:0/1419149819 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.813+0000 7f3cdb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cdc158be0 0x7f3cdc14d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:38008/0 (socket says 172.21.15.47:38008) 2024-04-24T22:40:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.814+0000 7f3cdb400640 1 -- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc0c1280 msgr2=0x7f3cdc160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.814+0000 7f3cdb400640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc0c1280 0x7f3cdc160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.814+0000 7f3cdb400640 1 -- 172.21.15.47:0/1419149819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cdc158fc0 msgr2=0x7f3cdc1f77c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:40:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.814+0000 7f3cdb400640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cdc158fc0 0x7f3cdc1f77c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.814+0000 7f3cdb400640 1 -- 172.21.15.47:0/1419149819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc1f82f0 con 0x7f3cdc158be0 2024-04-24T22:40:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.814+0000 7f3cdbe00640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc0c1280 0x7f3cdc160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.814+0000 7f3cdb400640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cdc158be0 0x7f3cdc14d690 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0b21b0 tx=0x7f3cc80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.815+0000 7f3cd3e00640 1 -- 172.21.15.47:0/1419149819 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8041070 con 0x7f3cdc158be0 2024-04-24T22:40:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.815+0000 7f3cd3e00640 1 -- 172.21.15.47:0/1419149819 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc803c870 con 0x7f3cdc158be0 2024-04-24T22:40:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.815+0000 7f3cd3e00640 1 -- 172.21.15.47:0/1419149819 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8037040 con 0x7f3cdc158be0 2024-04-24T22:40:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.815+0000 7f3ce2a00640 1 -- 172.21.15.47:0/1419149819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cdc161810 con 0x7f3cdc158be0 2024-04-24T22:40:24.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.815+0000 7f3ce2a00640 1 -- 172.21.15.47:0/1419149819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cdc1f8590 con 0x7f3cdc158be0 2024-04-24T22:40:24.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.816+0000 7f3ce2a00640 1 -- 172.21.15.47:0/1419149819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cdc073b90 con 0x7f3cdc158be0 2024-04-24T22:40:24.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.817+0000 7f3cd3e00640 1 -- 172.21.15.47:0/1419149819 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cc8044070 con 0x7f3cdc158be0 2024-04-24T22:40:24.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.818+0000 7f3cd3e00640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3cbc06be60 0x7f3cbc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:24.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.818+0000 7f3cd3e00640 1 -- 172.21.15.47:0/1419149819 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f3cc80ab590 con 0x7f3cdc158be0 2024-04-24T22:40:24.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.819+0000 7f3cdbe00640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3cbc06be60 0x7f3cbc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:24.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.819+0000 7f3cdbe00640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3cbc06be60 0x7f3cbc06e320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009810 tx=0x7f3ccc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:24.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:24.823+0000 7f3cd3e00640 1 -- 172.21.15.47:0/1419149819 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3cc804b070 con 0x7f3cdc158be0 2024-04-24T22:40:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.106+0000 7f3ce2a00640 1 -- 172.21.15.47:0/1419149819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3cdc0c76d0 con 0x7f3cdc158be0 2024-04-24T22:40:25.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.107+0000 7f3cd3e00640 1 -- 172.21.15.47:0/1419149819 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f3cc8079ea0 con 0x7f3cdc158be0 2024-04-24T22:40:25.109 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":66,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 -- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3cbc06be60 msgr2=0x7f3cbc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3cbc06be60 0x7f3cbc06e320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009810 tx=0x7f3ccc009190 comp rx=0 tx=0).stop 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 -- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cdc158be0 msgr2=0x7f3cdc14d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cdc158be0 0x7f3cdc14d690 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0b21b0 tx=0x7f3cc80046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 -- 172.21.15.47:0/1419149819 shutdown_connections 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3cbc06be60 0x7f3cbc06e320 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cdc158fc0 0x7f3cdc1f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cdc158be0 0x7f3cdc14d690 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 --2- 172.21.15.47:0/1419149819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cdc0c1280 0x7f3cdc160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 -- 172.21.15.47:0/1419149819 >> 172.21.15.47:0/1419149819 conn(0x7f3cdc06e580 msgr2=0x7f3cdc06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:25.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 -- 172.21.15.47:0/1419149819 shutdown_connections 2024-04-24T22:40:25.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:25.110+0000 7f3ce2a00640 1 -- 172.21.15.47:0/1419149819 wait complete. 2024-04-24T22:40:28.131 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:40:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.346+0000 7f2e9e200640 1 Processor -- start 2024-04-24T22:40:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.346+0000 7f2e9e200640 1 -- start start 2024-04-24T22:40:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.348+0000 7f2e9e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e981411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.348+0000 7f2e9e200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e9806bb90 0x7f2e98141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.348+0000 7f2e9e200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e9815c3d0 con 0x7f2e98072c60 2024-04-24T22:40:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.348+0000 7f2e9e200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2e98158bf0 con 0x7f2e9806bb90 2024-04-24T22:40:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.348+0000 7f2e9e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2e9815c6b0 con 0x7f2e98073040 2024-04-24T22:40:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.348+0000 7f2e9ce00640 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f2e98072c60 legacy=0x7f2e980c2d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-24T22:40:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.348+0000 7f2e9d800640 1 -- >> v2:172.21.15.47:3300/0 conn(0x7f2e9806bb90 msgr2=0x7f2e98141730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.348+0000 7f2e9d800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2e9806bb90 0x7f2e98141730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.348+0000 7f2e8fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e981411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.348+0000 7f2e8fe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e981411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:49844/0 (socket says 172.21.15.47:49844) 2024-04-24T22:40:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.349+0000 7f2e8fe00640 1 -- 172.21.15.47:0/2612003616 learned_addr learned my addr 172.21.15.47:0/2612003616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.349+0000 7f2e8fe00640 1 -- 172.21.15.47:0/2612003616 >> v2:172.21.15.47:3300/0 conn(0x7f2e9806bb90 msgr2=0x7f2e98141730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:40:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.349+0000 7f2e8fe00640 1 --2- 172.21.15.47:0/2612003616 >> v2:172.21.15.47:3300/0 conn(0x7f2e9806bb90 0x7f2e98141730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.349+0000 7f2e8fe00640 1 -- 172.21.15.47:0/2612003616 >> v1:172.21.15.47:6789/0 conn(0x7f2e98072c60 legacy=0x7f2e980c2d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:40:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.349+0000 7f2e8fe00640 1 -- 172.21.15.47:0/2612003616 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e981585a0 con 0x7f2e98073040 2024-04-24T22:40:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.350+0000 7f2e8fe00640 1 --2- 172.21.15.47:0/2612003616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e981411f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009870 tx=0x7f2e8002a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81ad734f3e51e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.350+0000 7f2e97e00640 1 -- 172.21.15.47:0/2612003616 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80037070 con 0x7f2e98073040 2024-04-24T22:40:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.350+0000 7f2e97e00640 1 -- 172.21.15.47:0/2612003616 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e80032440 con 0x7f2e98073040 2024-04-24T22:40:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.350+0000 7f2e97e00640 1 -- 172.21.15.47:0/2612003616 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80032810 con 0x7f2e98073040 2024-04-24T22:40:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.351+0000 7f2e9e200640 1 -- 172.21.15.47:0/2612003616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 msgr2=0x7f2e981411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.351+0000 7f2e9e200640 1 --2- 172.21.15.47:0/2612003616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e981411f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009870 tx=0x7f2e8002a2c0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.351+0000 7f2e9e200640 1 -- 172.21.15.47:0/2612003616 shutdown_connections 2024-04-24T22:40:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.351+0000 7f2e9e200640 1 --2- 172.21.15.47:0/2612003616 >> v2:172.21.15.47:3300/0 conn(0x7f2e9806bb90 0x7f2e98141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.351+0000 7f2e9e200640 1 --2- 172.21.15.47:0/2612003616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e981411f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.351+0000 7f2e9e200640 1 -- 172.21.15.47:0/2612003616 >> 172.21.15.47:0/2612003616 conn(0x7f2e980cb900 msgr2=0x7f2e980cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.351+0000 7f2e9e200640 1 -- 172.21.15.47:0/2612003616 shutdown_connections 2024-04-24T22:40:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.351+0000 7f2e9e200640 1 -- 172.21.15.47:0/2612003616 wait complete. 2024-04-24T22:40:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.353+0000 7f2e9e200640 1 Processor -- start 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.353+0000 7f2e9e200640 1 -- start start 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e9e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9806bb90 0x7f2e981548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e9e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e98072c60 0x7f2e98153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e9e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e9813a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e9e200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98159080 con 0x7f2e98072c60 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e9e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98158270 con 0x7f2e9806bb90 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e9e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2e98159200 con 0x7f2e98073040 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e8fe00640 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e98072c60 msgr2=0x7f2e98153910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e8fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e98072c60 0x7f2e98153910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e9d800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e9813a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e9ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9806bb90 0x7f2e981548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e9d800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e9813a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:49860/0 (socket says 172.21.15.47:49860) 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.354+0000 7f2e9d800640 1 -- 172.21.15.47:0/1235782103 learned_addr learned my addr 172.21.15.47:0/1235782103 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e9ce00640 1 -- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 msgr2=0x7f2e9813a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e9ce00640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e9813a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e9ce00640 1 -- 172.21.15.47:0/1235782103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e98072c60 msgr2=0x7f2e98153910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:40:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e9ce00640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e98072c60 0x7f2e98153910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e9ce00640 1 -- 172.21.15.47:0/1235782103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e98152ec0 con 0x7f2e9806bb90 2024-04-24T22:40:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e9d800640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e9813a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e9ce00640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9806bb90 0x7f2e981548d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2e88003980 tx=0x7f2e8800b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e95600640 1 -- 172.21.15.47:0/1235782103 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88003dc0 con 0x7f2e9806bb90 2024-04-24T22:40:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e95600640 1 -- 172.21.15.47:0/1235782103 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e88015070 con 0x7f2e9806bb90 2024-04-24T22:40:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e95600640 1 -- 172.21.15.47:0/1235782103 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88005290 con 0x7f2e9806bb90 2024-04-24T22:40:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e9e200640 1 -- 172.21.15.47:0/1235782103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9813cad0 con 0x7f2e9806bb90 2024-04-24T22:40:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.355+0000 7f2e9e200640 1 -- 172.21.15.47:0/1235782103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e9813a240 con 0x7f2e9806bb90 2024-04-24T22:40:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.356+0000 7f2e9e200640 1 -- 172.21.15.47:0/1235782103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e98068d50 con 0x7f2e9806bb90 2024-04-24T22:40:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.358+0000 7f2e95600640 1 -- 172.21.15.47:0/1235782103 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e8800f070 con 0x7f2e9806bb90 2024-04-24T22:40:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.359+0000 7f2e95600640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2e7406be10 0x7f2e7406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.359+0000 7f2e95600640 1 -- 172.21.15.47:0/1235782103 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f2e8808a710 con 0x7f2e9806bb90 2024-04-24T22:40:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.359+0000 7f2e8fe00640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2e7406be10 0x7f2e7406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.360+0000 7f2e8fe00640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2e7406be10 0x7f2e7406e2d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2e80034000 tx=0x7f2e8002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:28.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.363+0000 7f2e95600640 1 -- 172.21.15.47:0/1235782103 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e8800d040 con 0x7f2e9806bb90 2024-04-24T22:40:28.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.635+0000 7f2e9e200640 1 -- 172.21.15.47:0/1235782103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2e9804f5e0 con 0x7f2e9806bb90 2024-04-24T22:40:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.636+0000 7f2e95600640 1 -- 172.21.15.47:0/1235782103 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f2e88058fa0 con 0x7f2e9806bb90 2024-04-24T22:40:28.638 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":70,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:40:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.639+0000 7f2e9e200640 1 -- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2e7406be10 msgr2=0x7f2e7406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.639+0000 7f2e9e200640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2e7406be10 0x7f2e7406e2d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2e80034000 tx=0x7f2e8002e000 comp rx=0 tx=0).stop 2024-04-24T22:40:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.639+0000 7f2e9e200640 1 -- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9806bb90 msgr2=0x7f2e981548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.639+0000 7f2e9e200640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9806bb90 0x7f2e981548d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2e88003980 tx=0x7f2e8800b9b0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.639+0000 7f2e9e200640 1 -- 172.21.15.47:0/1235782103 shutdown_connections 2024-04-24T22:40:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.639+0000 7f2e9e200640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2e7406be10 0x7f2e7406e2d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.639+0000 7f2e9e200640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e98073040 0x7f2e9813a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.639+0000 7f2e9e200640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e98072c60 0x7f2e98153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.639+0000 7f2e9e200640 1 --2- 172.21.15.47:0/1235782103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9806bb90 0x7f2e981548d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:28.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.639+0000 7f2e9e200640 1 -- 172.21.15.47:0/1235782103 >> 172.21.15.47:0/1235782103 conn(0x7f2e980cb900 msgr2=0x7f2e9813d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:28.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.640+0000 7f2e9e200640 1 -- 172.21.15.47:0/1235782103 shutdown_connections 2024-04-24T22:40:28.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:28.640+0000 7f2e9e200640 1 -- 172.21.15.47:0/1235782103 wait complete. 2024-04-24T22:40:31.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:40:31.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.875+0000 7fd161e00640 1 Processor -- start 2024-04-24T22:40:31.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.875+0000 7fd161e00640 1 -- start start 2024-04-24T22:40:31.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd161e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd15c072c60 0x7fd15c0c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:31.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd161e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd15c06bb90 0x7fd15c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:31.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd161e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd15c15c3d0 con 0x7fd15c073040 2024-04-24T22:40:31.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd161e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd15c158bf0 con 0x7fd15c06bb90 2024-04-24T22:40:31.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd161e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd15c15c6b0 con 0x7fd15c072c60 2024-04-24T22:40:31.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd161400640 1 -- >> v2:172.21.15.47:3300/0 conn(0x7fd15c06bb90 msgr2=0x7fd15c15b860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:31.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd161400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd15c06bb90 0x7fd15c15b860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:31.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd160a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd15c072c60 0x7fd15c0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:31.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd160a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd15c072c60 0x7fd15c0c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:34514/0 (socket says 172.21.15.47:34514) 2024-04-24T22:40:31.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd15be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd15c073040 0x7fd15c1411f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:34156/0 (socket says 172.21.15.47:34156) 2024-04-24T22:40:31.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.877+0000 7fd160a00640 1 -- 172.21.15.47:0/1718052238 learned_addr learned my addr 172.21.15.47:0/1718052238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:31.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.878+0000 7fd160a00640 1 -- 172.21.15.47:0/1718052238 >> v2:172.21.15.47:3300/0 conn(0x7fd15c06bb90 msgr2=0x7fd15c15b860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:40:31.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.878+0000 7fd160a00640 1 --2- 172.21.15.47:0/1718052238 >> v2:172.21.15.47:3300/0 conn(0x7fd15c06bb90 0x7fd15c15b860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:31.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.878+0000 7fd160a00640 1 -- 172.21.15.47:0/1718052238 >> v1:172.21.15.113:6789/0 conn(0x7fd15c073040 legacy=0x7fd15c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:31.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.878+0000 7fd160a00640 1 -- 172.21.15.47:0/1718052238 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c1585a0 con 0x7fd15c072c60 2024-04-24T22:40:31.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.879+0000 7fd160a00640 1 --2- 172.21.15.47:0/1718052238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c072c60 0x7fd15c0c2d60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd15c06b280 tx=0x7fd15002a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffef3bf0e6d77f53 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:31.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.880+0000 7fd15b400640 1 -- 172.21.15.47:0/1718052238 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150037070 con 0x7fd15c072c60 2024-04-24T22:40:31.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.880+0000 7fd15b400640 1 -- 172.21.15.47:0/1718052238 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd150032440 con 0x7fd15c072c60 2024-04-24T22:40:31.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.880+0000 7fd15b400640 1 -- 172.21.15.47:0/1718052238 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1500327f0 con 0x7fd15c072c60 2024-04-24T22:40:31.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.880+0000 7fd161e00640 1 -- 172.21.15.47:0/1718052238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c072c60 msgr2=0x7fd15c0c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:31.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.880+0000 7fd161e00640 1 --2- 172.21.15.47:0/1718052238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c072c60 0x7fd15c0c2d60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd15c06b280 tx=0x7fd15002a290 comp rx=0 tx=0).stop 2024-04-24T22:40:31.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.881+0000 7fd161e00640 1 -- 172.21.15.47:0/1718052238 shutdown_connections 2024-04-24T22:40:31.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.881+0000 7fd161e00640 1 --2- 172.21.15.47:0/1718052238 >> v2:172.21.15.47:3300/0 conn(0x7fd15c06bb90 0x7fd15c15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:31.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.881+0000 7fd161e00640 1 --2- 172.21.15.47:0/1718052238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c072c60 0x7fd15c0c2d60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:31.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.881+0000 7fd161e00640 1 -- 172.21.15.47:0/1718052238 >> 172.21.15.47:0/1718052238 conn(0x7fd15c0cb900 msgr2=0x7fd15c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:31.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.881+0000 7fd161e00640 1 -- 172.21.15.47:0/1718052238 shutdown_connections 2024-04-24T22:40:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.881+0000 7fd161e00640 1 -- 172.21.15.47:0/1718052238 wait complete. 2024-04-24T22:40:31.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.882+0000 7fd161e00640 1 Processor -- start 2024-04-24T22:40:31.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.883+0000 7fd161e00640 1 -- start start 2024-04-24T22:40:31.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.883+0000 7fd161e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c06bb90 0x7fd15c0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:31.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd161e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd15c072c60 0x7fd15c1e17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:31.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd161e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c073040 0x7fd15c0c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:31.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd161e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c0c32a0 con 0x7fd15c073040 2024-04-24T22:40:31.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd161e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c158270 con 0x7fd15c06bb90 2024-04-24T22:40:31.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd161e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd15c159080 con 0x7fd15c072c60 2024-04-24T22:40:31.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd161400640 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c073040 msgr2=0x7fd15c0c6ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd161400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c073040 0x7fd15c0c6ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd15be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd15c072c60 0x7fd15c1e17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd160a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c06bb90 0x7fd15c0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd160a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c06bb90 0x7fd15c0ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:34516/0 (socket says 172.21.15.47:34516) 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd15be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd15c072c60 0x7fd15c1e17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:49870/0 (socket says 172.21.15.47:49870) 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd160a00640 1 -- 172.21.15.47:0/3334291893 learned_addr learned my addr 172.21.15.47:0/3334291893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd160a00640 1 -- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd15c072c60 msgr2=0x7fd15c1e17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd160a00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd15c072c60 0x7fd15c1e17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd160a00640 1 -- 172.21.15.47:0/3334291893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c073040 msgr2=0x7fd15c0c6ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd160a00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c073040 0x7fd15c0c6ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd160a00640 1 -- 172.21.15.47:0/3334291893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c0ca7c0 con 0x7fd15c06bb90 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.884+0000 7fd15be00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd15c072c60 0x7fd15c1e17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:31.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.885+0000 7fd160a00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c06bb90 0x7fd15c0ca280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd1500090b0 tx=0x7fd1500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.885+0000 7fd158c00640 1 -- 172.21.15.47:0/3334291893 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150041070 con 0x7fd15c06bb90 2024-04-24T22:40:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.885+0000 7fd158c00640 1 -- 172.21.15.47:0/3334291893 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15003c870 con 0x7fd15c06bb90 2024-04-24T22:40:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.885+0000 7fd158c00640 1 -- 172.21.15.47:0/3334291893 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150037040 con 0x7fd15c06bb90 2024-04-24T22:40:31.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.886+0000 7fd161e00640 1 -- 172.21.15.47:0/3334291893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c1e3580 con 0x7fd15c06bb90 2024-04-24T22:40:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.886+0000 7fd161e00640 1 -- 172.21.15.47:0/3334291893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c1e3b00 con 0x7fd15c06bb90 2024-04-24T22:40:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.887+0000 7fd161e00640 1 -- 172.21.15.47:0/3334291893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15c068d50 con 0x7fd15c06bb90 2024-04-24T22:40:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.888+0000 7fd158c00640 1 -- 172.21.15.47:0/3334291893 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd150044070 con 0x7fd15c06bb90 2024-04-24T22:40:31.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.889+0000 7fd158c00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd13c06be60 0x7fd13c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.889+0000 7fd158c00640 1 -- 172.21.15.47:0/3334291893 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd1500abdc0 con 0x7fd15c06bb90 2024-04-24T22:40:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.890+0000 7fd15be00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd13c06be60 0x7fd13c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.890+0000 7fd15be00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd13c06be60 0x7fd13c06e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd15c140770 tx=0x7fd148007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:31.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:31.893+0000 7fd158c00640 1 -- 172.21.15.47:0/3334291893 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd15007a230 con 0x7fd15c06bb90 2024-04-24T22:40:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.166+0000 7fd161e00640 1 -- 172.21.15.47:0/3334291893 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd15c04f5e0 con 0x7fd15c06bb90 2024-04-24T22:40:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.167+0000 7fd158c00640 1 -- 172.21.15.47:0/3334291893 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7fd15007e530 con 0x7fd15c06bb90 2024-04-24T22:40:32.169 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":73,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 -- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd13c06be60 msgr2=0x7fd13c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd13c06be60 0x7fd13c06e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd15c140770 tx=0x7fd148007e80 comp rx=0 tx=0).stop 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 -- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c06bb90 msgr2=0x7fd15c0ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c06bb90 0x7fd15c0ca280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd1500090b0 tx=0x7fd1500046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 -- 172.21.15.47:0/3334291893 shutdown_connections 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd13c06be60 0x7fd13c06e320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c073040 0x7fd15c0c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd15c072c60 0x7fd15c1e17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 --2- 172.21.15.47:0/3334291893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15c06bb90 0x7fd15c0ca280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 -- 172.21.15.47:0/3334291893 >> 172.21.15.47:0/3334291893 conn(0x7fd15c0cb900 msgr2=0x7fd15c13e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 -- 172.21.15.47:0/3334291893 shutdown_connections 2024-04-24T22:40:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:32.170+0000 7fd161e00640 1 -- 172.21.15.47:0/3334291893 wait complete. 2024-04-24T22:40:35.190 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:40:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.407+0000 7f4ad9a00640 1 Processor -- start 2024-04-24T22:40:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.407+0000 7f4ad9a00640 1 -- start start 2024-04-24T22:40:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.409+0000 7f4ad9a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.409+0000 7f4ad9a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.409+0000 7f4ad9a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad415c3d0 con 0x7f4ad4072c60 2024-04-24T22:40:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.409+0000 7f4ad9a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad4158bf0 con 0x7f4ad406bb90 2024-04-24T22:40:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.409+0000 7f4ad9a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad415c6b0 con 0x7f4ad4073040 2024-04-24T22:40:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad3e00640 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f4ad4072c60 legacy=0x7f4ad40c2d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-24T22:40:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad9000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad9000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:34524/0 (socket says 172.21.15.47:34524) 2024-04-24T22:40:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad9000640 1 -- 172.21.15.47:0/3809398549 learned_addr learned my addr 172.21.15.47:0/3809398549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad9000640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 msgr2=0x7f4ad4141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T22:40:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad9000640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 msgr2=0x7f4ad4141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:40:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:40:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 msgr2=0x7f4ad41411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:40:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 msgr2=0x7f4ad41411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:40:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:40:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.410+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:35.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.611+0000 7f4ad3e00640 1 -- 172.21.15.47:0/3809398549 >> v1:172.21.15.47:6789/0 conn(0x7f4ad4072c60 legacy=0x7f4ad40c2d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-24T22:40:35.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.611+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.611+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:35.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.611+0000 7f4ad9000640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 msgr2=0x7f4ad4141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:40:35.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.611+0000 7f4ad9000640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 msgr2=0x7f4ad4141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:40:35.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.611+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:40:35.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.611+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:40:35.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.611+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 msgr2=0x7f4ad41411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T22:40:35.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.612+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 msgr2=0x7f4ad41411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:40:35.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.612+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:40:35.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:35.612+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:40:36.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.012+0000 7f4ad3e00640 1 -- 172.21.15.47:0/3809398549 >> v1:172.21.15.47:6789/0 conn(0x7f4ad4072c60 legacy=0x7f4ad40c2d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-24T22:40:36.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.012+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.013+0000 7f4ad9000640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 msgr2=0x7f4ad4141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:40:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.013+0000 7f4ad9000640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 msgr2=0x7f4ad4141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:40:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.013+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:40:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.013+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:40:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.013+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.013+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 msgr2=0x7f4ad41411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:40:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.013+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 msgr2=0x7f4ad41411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:40:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.013+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:40:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.014+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:40:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.813+0000 7f4ad3e00640 1 -- 172.21.15.47:0/3809398549 >> v1:172.21.15.47:6789/0 conn(0x7f4ad4072c60 legacy=0x7f4ad40c2d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-24T22:40:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.814+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.814+0000 7f4ad9000640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 msgr2=0x7f4ad4141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:40:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.814+0000 7f4ad9000640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 msgr2=0x7f4ad4141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:40:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.814+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:40:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.814+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:40:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.815+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.815+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 msgr2=0x7f4ad41411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:40:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.815+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 msgr2=0x7f4ad41411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:40:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.815+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:40:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:36.815+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:40:38.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.410+0000 7f4ad1600640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 msgr2=0x7f4ad41411f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:40:38.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.410+0000 7f4ad1600640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ad4073040 0x7f4ad41411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.410+0000 7f4ad1600640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 msgr2=0x7f4ad4141730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:40:38.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.410+0000 7f4ad1600640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.113:3300/0 conn(0x7f4ad406bb90 0x7f4ad4141730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.410+0000 7f4ad1600640 1 -- 172.21.15.47:0/3809398549 >> v1:172.21.15.47:6789/0 conn(0x7f4ad4072c60 legacy=0x7f4ad40c2d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:40:38.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.410+0000 7f4ad1600640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.47:3300/0 conn(0x7f4ab40073c0 0x7f4ab4009880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:38.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.410+0000 7f4ad1600640 1 -- 172.21.15.47:0/3809398549 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad41585a0 con 0x7f4ab4003680 2024-04-24T22:40:38.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.410+0000 7f4ad1600640 1 -- 172.21.15.47:0/3809398549 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad4158bf0 con 0x7f4ab40073c0 2024-04-24T22:40:38.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.410+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.47:3300/0 conn(0x7f4ab40073c0 msgr2=0x7f4ab4009880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:38.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.410+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.47:3300/0 conn(0x7f4ab40073c0 0x7f4ab4009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.611+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.47:3300/0 conn(0x7f4ab40073c0 msgr2=0x7f4ab4009880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:38.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.611+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.47:3300/0 conn(0x7f4ab40073c0 0x7f4ab4009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:40:38.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.920+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2171838615 0 0) 0x7f4ad41585a0 con 0x7f4ab4003680 2024-04-24T22:40:38.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.920+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ab8003520 con 0x7f4ab4003680 2024-04-24T22:40:38.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2082603120 0 0) 0x7f4ab8003520 con 0x7f4ab4003680 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 >> v2:172.21.15.47:3300/0 conn(0x7f4ab40073c0 msgr2=0x7f4ab4009880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 --2- 172.21.15.47:0/3809398549 >> v2:172.21.15.47:3300/0 conn(0x7f4ab40073c0 0x7f4ab4009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad415c3d0 con 0x7f4ab4003680 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 567141988 0 0) 0x7f4ac4004240 con 0x7f4ab4003680 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 >> v1:172.21.15.113:6789/0 conn(0x7f4ab4003680 legacy=0x7f4ab4005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad9000640 1 -- 172.21.15.47:0/3809398549 reap_dead start 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 0x7f4ab8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8004800 0x7f4ab8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab80052e0 0x7f4ab800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4158bf0 con 0x7f4ab8004800 2024-04-24T22:40:38.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab800c230 con 0x7f4ab80039b0 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.924+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab800c430 con 0x7f4ab80052e0 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.925+0000 7f4ad3e00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 0x7f4ab8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.925+0000 7f4ad3400640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8004800 msgr2=0x7f4ab8004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.925+0000 7f4ad3400640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8004800 0x7f4ab8004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.925+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab80052e0 0x7f4ab800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.925+0000 7f4ad3e00640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab80052e0 msgr2=0x7f4ab800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.925+0000 7f4ad3e00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab80052e0 0x7f4ab800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.925+0000 7f4ad3e00640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8004800 msgr2=0x7f4ab8004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.925+0000 7f4ad3e00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8004800 0x7f4ab8004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.925+0000 7f4ad3e00640 1 -- 172.21.15.47:0/3809398549 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab8003520 con 0x7f4ab80039b0 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.925+0000 7f4ad9000640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab80052e0 0x7f4ab800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T22:40:38.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.927+0000 7f4ad3e00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 0x7f4ab8003db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4000c00 tx=0x7f4ab8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8acdf9ed86ad22d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:38.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.927+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4007e00 con 0x7f4ab80039b0 2024-04-24T22:40:38.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.928+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac4006290 con 0x7f4ab80039b0 2024-04-24T22:40:38.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.928+0000 7f4ad2a00640 1 -- 172.21.15.47:0/3809398549 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40046e0 con 0x7f4ab80039b0 2024-04-24T22:40:38.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.929+0000 7f4ad9a00640 1 -- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 msgr2=0x7f4ab8003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:38.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.929+0000 7f4ad9a00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 0x7f4ab8003db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4000c00 tx=0x7f4ab8005830 comp rx=0 tx=0).stop 2024-04-24T22:40:38.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.929+0000 7f4ad9a00640 1 -- 172.21.15.47:0/3809398549 shutdown_connections 2024-04-24T22:40:38.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.929+0000 7f4ad9a00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab80052e0 0x7f4ab800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.929+0000 7f4ad9a00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8004800 0x7f4ab8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.929+0000 7f4ad9a00640 1 --2- 172.21.15.47:0/3809398549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 0x7f4ab8003db0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.929+0000 7f4ad9a00640 1 -- 172.21.15.47:0/3809398549 >> 172.21.15.47:0/3809398549 conn(0x7f4ad40cb900 msgr2=0x7f4ad40cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:38.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.929+0000 7f4ad9a00640 1 -- 172.21.15.47:0/3809398549 shutdown_connections 2024-04-24T22:40:38.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.930+0000 7f4ad9a00640 1 -- 172.21.15.47:0/3809398549 wait complete. 2024-04-24T22:40:38.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.935+0000 7f4ad9a00640 1 Processor -- start 2024-04-24T22:40:38.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.935+0000 7f4ad9a00640 1 -- start start 2024-04-24T22:40:38.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.937+0000 7f4ad9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 0x7f4ad4152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:38.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad9a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab8004800 0x7f4ad4152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:38.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab80052e0 0x7f4ad413e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:38.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad9a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4159080 con 0x7f4ab80052e0 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad9a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4158270 con 0x7f4ab80039b0 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad9a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad4159200 con 0x7f4ab8004800 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad9000640 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab80052e0 msgr2=0x7f4ad413e030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab80052e0 0x7f4ad413e030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab8004800 0x7f4ad4152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab8004800 0x7f4ad4152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:33794/0 (socket says 172.21.15.47:33794) 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad3400640 1 -- 172.21.15.47:0/4217691789 learned_addr learned my addr 172.21.15.47:0/4217691789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.938+0000 7f4ad3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 0x7f4ad4152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.939+0000 7f4ad3400640 1 -- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 msgr2=0x7f4ad4152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.939+0000 7f4ad3400640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 0x7f4ad4152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.939+0000 7f4ad3400640 1 -- 172.21.15.47:0/4217691789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab80052e0 msgr2=0x7f4ad413e030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.939+0000 7f4ad3400640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab80052e0 0x7f4ad413e030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.939+0000 7f4ad3400640 1 -- 172.21.15.47:0/4217691789 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad4155bb0 con 0x7f4ab8004800 2024-04-24T22:40:38.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.939+0000 7f4ad3400640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab8004800 0x7f4ad4152f40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0013610 tx=0x7f4ac0013ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.939+0000 7f4acbe00640 1 -- 172.21.15.47:0/4217691789 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac001c070 con 0x7f4ab8004800 2024-04-24T22:40:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.940+0000 7f4acbe00640 1 -- 172.21.15.47:0/4217691789 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac00158b0 con 0x7f4ab8004800 2024-04-24T22:40:38.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.940+0000 7f4ad9a00640 1 -- 172.21.15.47:0/4217691789 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad41543e0 con 0x7f4ab8004800 2024-04-24T22:40:38.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.940+0000 7f4acbe00640 1 -- 172.21.15.47:0/4217691789 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0015ca0 con 0x7f4ab8004800 2024-04-24T22:40:38.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.940+0000 7f4ad9a00640 1 -- 172.21.15.47:0/4217691789 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad41549a0 con 0x7f4ab8004800 2024-04-24T22:40:38.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.942+0000 7f4ad9a00640 1 -- 172.21.15.47:0/4217691789 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad4068d50 con 0x7f4ab8004800 2024-04-24T22:40:38.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.942+0000 7f4acbe00640 1 -- 172.21.15.47:0/4217691789 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ac001f070 con 0x7f4ab8004800 2024-04-24T22:40:38.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.944+0000 7f4acbe00640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4ab406c4e0 0x7f4ab406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:38.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.944+0000 7f4acbe00640 1 -- 172.21.15.47:0/4217691789 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f4ac008b3c0 con 0x7f4ab8004800 2024-04-24T22:40:38.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.944+0000 7f4ad3e00640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4ab406c4e0 0x7f4ab406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:38.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.945+0000 7f4ad3e00640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4ab406c4e0 0x7f4ab406e9a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4033000 tx=0x7f4ac402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:38.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:38.949+0000 7f4acbe00640 1 -- 172.21.15.47:0/4217691789 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ac0059580 con 0x7f4ab8004800 2024-04-24T22:40:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.245+0000 7f4ad9a00640 1 -- 172.21.15.47:0/4217691789 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4ad413fde0 con 0x7f4ab8004800 2024-04-24T22:40:39.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.246+0000 7f4acbe00640 1 -- 172.21.15.47:0/4217691789 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f4ac005d880 con 0x7f4ab8004800 2024-04-24T22:40:39.254 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":6,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:40:39.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.249+0000 7f4ad9a00640 1 -- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4ab406c4e0 msgr2=0x7f4ab406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.249+0000 7f4ad9a00640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4ab406c4e0 0x7f4ab406e9a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4033000 tx=0x7f4ac402e000 comp rx=0 tx=0).stop 2024-04-24T22:40:39.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.249+0000 7f4ad9a00640 1 -- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab8004800 msgr2=0x7f4ad4152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.249+0000 7f4ad9a00640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab8004800 0x7f4ad4152f40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0013610 tx=0x7f4ac0013ad0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.249+0000 7f4ad9a00640 1 -- 172.21.15.47:0/4217691789 shutdown_connections 2024-04-24T22:40:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.249+0000 7f4ad9a00640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4ab406c4e0 0x7f4ab406e9a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.249+0000 7f4ad9a00640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab80052e0 0x7f4ad413e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.249+0000 7f4ad9a00640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ab8004800 0x7f4ad4152f40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.249+0000 7f4ad9a00640 1 --2- 172.21.15.47:0/4217691789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ab80039b0 0x7f4ad4152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.249+0000 7f4ad9a00640 1 -- 172.21.15.47:0/4217691789 >> 172.21.15.47:0/4217691789 conn(0x7f4ad40cb900 msgr2=0x7f4ad40c53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.250+0000 7f4ad9a00640 1 -- 172.21.15.47:0/4217691789 shutdown_connections 2024-04-24T22:40:39.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.250+0000 7f4ad9a00640 1 -- 172.21.15.47:0/4217691789 wait complete. 2024-04-24T22:40:39.269 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-24T22:40:39.270 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-24T22:40:39.270 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:40:39.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.479+0000 7f2162400640 1 Processor -- start 2024-04-24T22:40:39.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.479+0000 7f2162400640 1 -- start start 2024-04-24T22:40:39.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.481+0000 7f2162400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f215c0cbb40 0x7f215c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.481+0000 7f2162400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f215c073280 0x7f215c0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.481+0000 7f2162400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f215c15b2c0 con 0x7f215c154560 2024-04-24T22:40:39.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.481+0000 7f2162400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f215c15da90 con 0x7f215c0cbb40 2024-04-24T22:40:39.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.481+0000 7f2162400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f215c15dd70 con 0x7f215c073280 2024-04-24T22:40:39.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.481+0000 7f215be00640 1 -- >> v2:172.21.15.47:3300/0 conn(0x7f215c0cbb40 msgr2=0x7f215c0cc660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:39.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.481+0000 7f215be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f215c0cbb40 0x7f215c0cc660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:39.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.481+0000 7f2161a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f215c073280 0x7f215c0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.481+0000 7f2161000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f215c154560 0x7f215c0cb430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:43066/0 (socket says 172.21.15.47:43066) 2024-04-24T22:40:39.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.481+0000 7f2161000640 1 -- 172.21.15.47:0/2811238527 learned_addr learned my addr 172.21.15.47:0/2811238527 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:39.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.482+0000 7f215b400640 1 -- 172.21.15.47:0/2811238527 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2918542943 0 0) 0x7f215c15b2c0 con 0x7f215c154560 2024-04-24T22:40:39.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.482+0000 7f215b400640 1 -- 172.21.15.47:0/2811238527 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2140003520 con 0x7f215c154560 2024-04-24T22:40:39.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.482+0000 7f2161a00640 1 -- 172.21.15.47:0/2811238527 >> v2:172.21.15.47:3300/0 conn(0x7f215c0cbb40 msgr2=0x7f215c0cc660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:40:39.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.482+0000 7f2161a00640 1 --2- 172.21.15.47:0/2811238527 >> v2:172.21.15.47:3300/0 conn(0x7f215c0cbb40 0x7f215c0cc660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.482+0000 7f2161a00640 1 -- 172.21.15.47:0/2811238527 >> v1:172.21.15.113:6789/0 conn(0x7f215c154560 legacy=0x7f215c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.482+0000 7f2161a00640 1 -- 172.21.15.47:0/2811238527 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c15e050 con 0x7f215c073280 2024-04-24T22:40:39.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.483+0000 7f2161a00640 1 --2- 172.21.15.47:0/2811238527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c073280 0x7f215c0ccba0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f215c0724d0 tx=0x7f215402a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=718f2d6917a19814 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:39.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.484+0000 7f215b400640 1 -- 172.21.15.47:0/2811238527 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154007740 con 0x7f215c073280 2024-04-24T22:40:39.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.484+0000 7f215b400640 1 -- 172.21.15.47:0/2811238527 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21540078e0 con 0x7f215c073280 2024-04-24T22:40:39.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.484+0000 7f215b400640 1 -- 172.21.15.47:0/2811238527 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154007cb0 con 0x7f215c073280 2024-04-24T22:40:39.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.484+0000 7f2162400640 1 -- 172.21.15.47:0/2811238527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c073280 msgr2=0x7f215c0ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.484+0000 7f2162400640 1 --2- 172.21.15.47:0/2811238527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c073280 0x7f215c0ccba0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f215c0724d0 tx=0x7f215402a610 comp rx=0 tx=0).stop 2024-04-24T22:40:39.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.484+0000 7f2162400640 1 -- 172.21.15.47:0/2811238527 shutdown_connections 2024-04-24T22:40:39.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.484+0000 7f2162400640 1 --2- 172.21.15.47:0/2811238527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c073280 0x7f215c0ccba0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.484+0000 7f2162400640 1 --2- 172.21.15.47:0/2811238527 >> v2:172.21.15.47:3300/0 conn(0x7f215c0cbb40 0x7f215c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.484+0000 7f2162400640 1 -- 172.21.15.47:0/2811238527 >> 172.21.15.47:0/2811238527 conn(0x7f215c0672c0 msgr2=0x7f215c072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:39.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.485+0000 7f2162400640 1 -- 172.21.15.47:0/2811238527 shutdown_connections 2024-04-24T22:40:39.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.485+0000 7f2162400640 1 -- 172.21.15.47:0/2811238527 wait complete. 2024-04-24T22:40:39.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.486+0000 7f2162400640 1 Processor -- start 2024-04-24T22:40:39.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.487+0000 7f2162400640 1 -- start start 2024-04-24T22:40:39.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f2162400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215c073280 0x7f215c150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f2162400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215c0cbb40 0x7f215c0b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f2162400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c154560 0x7f215c0b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f2162400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f215c15ae80 con 0x7f215c073280 2024-04-24T22:40:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f2162400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f215c0caad0 con 0x7f215c154560 2024-04-24T22:40:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f2162400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f215c15b000 con 0x7f215c0cbb40 2024-04-24T22:40:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f2161000640 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215c073280 msgr2=0x7f215c150950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f2161000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215c073280 0x7f215c150950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:39.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f215be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215c0cbb40 0x7f215c0b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f2161a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c154560 0x7f215c0b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f215be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215c0cbb40 0x7f215c0b3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:33800/0 (socket says 172.21.15.47:33800) 2024-04-24T22:40:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.488+0000 7f215be00640 1 -- 172.21.15.47:0/1919124001 learned_addr learned my addr 172.21.15.47:0/1919124001 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f215be00640 1 -- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c154560 msgr2=0x7f215c0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f215be00640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c154560 0x7f215c0b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f215be00640 1 -- 172.21.15.47:0/1919124001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215c073280 msgr2=0x7f215c150950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:40:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f215be00640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f215c073280 0x7f215c150950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f215be00640 1 -- 172.21.15.47:0/1919124001 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c0b4c70 con 0x7f215c0cbb40 2024-04-24T22:40:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f215be00640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215c0cbb40 0x7f215c0b3070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f214800b550 tx=0x7f214800ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148013020 con 0x7f215c0cbb40 2024-04-24T22:40:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2148003de0 con 0x7f215c0cbb40 2024-04-24T22:40:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21480052b0 con 0x7f215c0cbb40 2024-04-24T22:40:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f2162400640 1 -- 172.21.15.47:0/1919124001 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c1f02d0 con 0x7f215c0cbb40 2024-04-24T22:40:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.489+0000 7f2162400640 1 -- 172.21.15.47:0/1919124001 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c1f0890 con 0x7f215c0cbb40 2024-04-24T22:40:39.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.490+0000 7f2162400640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c15a500 0x7f215c0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.491+0000 7f2162400640 1 -- 172.21.15.47:0/1919124001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f215c15d620 con 0x7f215c15a500 2024-04-24T22:40:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.491+0000 7f2161000640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c15a500 0x7f215c0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.491+0000 7f2161000640 1 -- 172.21.15.47:0/1919124001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f215c0c6080 con 0x7f215c15a500 2024-04-24T22:40:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.491+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2148016020 con 0x7f215c0cbb40 2024-04-24T22:40:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.492+0000 7f2161000640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c15a500 0x7f215c0b4690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f214c004810 tx=0x7f214c00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:39.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.492+0000 7f2158c00640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f213c06be10 0x7f213c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.492+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f214808acf0 con 0x7f215c0cbb40 2024-04-24T22:40:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.493+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c017070 con 0x7f215c15a500 2024-04-24T22:40:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.493+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f214c005ce0 con 0x7f215c15a500 2024-04-24T22:40:39.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.493+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c15a500 msgr2=0x7f215c0b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.493+0000 7f2158c00640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c15a500 0x7f215c0b4690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f214c004810 tx=0x7f214c00c810 comp rx=0 tx=0).stop 2024-04-24T22:40:39.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.493+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c15a500 msgr2=0x7f215c0b4690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:40:39.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.493+0000 7f2158c00640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c15a500 0x7f215c0b4690 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f214c004810 tx=0x7f214c00c810 comp rx=0 tx=0).stop 2024-04-24T22:40:39.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.493+0000 7f2161a00640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f213c06be10 0x7f213c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.493+0000 7f2161a00640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f213c06be10 0x7f213c06e2d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2154034000 tx=0x7f215402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:39.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.500+0000 7f2162400640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c1f0a90 0x7f215c145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.500+0000 7f2162400640 1 -- 172.21.15.47:0/1919124001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f215c0ca080 con 0x7f215c1f0a90 2024-04-24T22:40:39.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.501+0000 7f2161000640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c1f0a90 0x7f215c145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.501+0000 7f2161000640 1 -- 172.21.15.47:0/1919124001 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f215c0cd3d0 con 0x7f215c1f0a90 2024-04-24T22:40:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.501+0000 7f2161000640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c1f0a90 0x7f215c145220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f215c0ca480 tx=0x7f214c000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.502+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c017040 con 0x7f215c1f0a90 2024-04-24T22:40:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.502+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5054 (secure 0 0 0) 0x7f214c012400 con 0x7f215c1f0a90 2024-04-24T22:40:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.502+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c1f0a90 msgr2=0x7f215c145220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.502+0000 7f2158c00640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c1f0a90 0x7f215c145220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f215c0ca480 tx=0x7f214c000ed0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.502+0000 7f2158c00640 1 -- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c1f0a90 msgr2=0x7f215c145220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:40:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.502+0000 7f2158c00640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f215c1f0a90 0x7f215c145220 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f215c0ca480 tx=0x7f214c000ed0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.507 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:40:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:40:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:40:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:40:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 6, 2024-04-24T22:40:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 81069, 2024-04-24T22:40:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:40:39.507 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:39.508 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:40:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:40:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:39.509 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:40:39.510 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:40:39.511 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:40:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:40:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:40:39.513 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.514 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:39.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:40:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:40:39.516 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:39.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:39.519 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:40:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.521 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:40:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.522 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:40:39.522 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:40:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.503+0000 7f2162400640 1 -- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f213c06be10 msgr2=0x7f213c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.503+0000 7f2162400640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f213c06be10 0x7f213c06e2d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2154034000 tx=0x7f215402f000 comp rx=0 tx=0).stop 2024-04-24T22:40:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.503+0000 7f2162400640 1 -- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215c0cbb40 msgr2=0x7f215c0b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.503+0000 7f2162400640 1 --2- 172.21.15.47:0/1919124001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f215c0cbb40 0x7f215c0b3070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f214800b550 tx=0x7f214800ba10 comp rx=0 tx=0).stop 2024-04-24T22:40:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.503+0000 7f2161a00640 1 -- 172.21.15.47:0/1919124001 reap_dead start 2024-04-24T22:40:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.503+0000 7f2161a00640 1 -- 172.21.15.47:0/1919124001 reap_dead start 2024-04-24T22:40:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.503+0000 7f2162400640 1 -- 172.21.15.47:0/1919124001 shutdown_connections 2024-04-24T22:40:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.503+0000 7f2162400640 1 -- 172.21.15.47:0/1919124001 >> 172.21.15.47:0/1919124001 conn(0x7f215c0672c0 msgr2=0x7f215c152a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.503+0000 7f2162400640 1 -- 172.21.15.47:0/1919124001 shutdown_connections 2024-04-24T22:40:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.503+0000 7f2162400640 1 -- 172.21.15.47:0/1919124001 wait complete. 2024-04-24T22:40:39.525 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-24T22:40:39.525 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-24T22:40:39.525 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-24T22:40:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.729+0000 7f2b6be00640 1 Processor -- start 2024-04-24T22:40:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.729+0000 7f2b6be00640 1 -- start start 2024-04-24T22:40:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.730+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1514f0 0x7f2b6c06e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.730+0000 7f2b6be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b6c0c6a20 0x7f2b6c0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6be00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b6c071570 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6be00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2b6c15b470 con 0x7f2b6c0c6a20 2024-04-24T22:40:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6be00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c15a8a0 con 0x7f2b6c1514f0 2024-04-24T22:40:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b6c0c6a20 0x7f2b6c0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6a000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2b6c06e6f0 0x7f2b6c0c6310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:43082/0 (socket says 172.21.15.47:43082) 2024-04-24T22:40:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6a000640 1 -- 172.21.15.47:0/3861169687 learned_addr learned my addr 172.21.15.47:0/3861169687 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1514f0 0x7f2b6c06e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b69600640 1 -- 172.21.15.47:0/3861169687 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547286418 0 0) 0x7f2b6c071570 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b69600640 1 -- 172.21.15.47:0/3861169687 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b50003520 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6aa00640 1 -- 172.21.15.47:0/3861169687 >> v2:172.21.15.113:3300/0 conn(0x7f2b6c0c6a20 msgr2=0x7f2b6c0707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:40:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6aa00640 1 --2- 172.21.15.47:0/3861169687 >> v2:172.21.15.113:3300/0 conn(0x7f2b6c0c6a20 0x7f2b6c0707e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6aa00640 1 -- 172.21.15.47:0/3861169687 >> v1:172.21.15.113:6789/0 conn(0x7f2b6c06e6f0 legacy=0x7f2b6c0c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6aa00640 1 -- 172.21.15.47:0/3861169687 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c1518d0 con 0x7f2b6c1514f0 2024-04-24T22:40:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.731+0000 7f2b6b400640 1 --2- 172.21.15.47:0/3861169687 >> v2:172.21.15.113:3300/0 conn(0x7f2b6c0c6a20 0x7f2b6c0707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:40:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.732+0000 7f2b6aa00640 1 --2- 172.21.15.47:0/3861169687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1514f0 0x7f2b6c06e1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0097a0 tx=0x7f2b5c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4445aeae6488d4ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.733+0000 7f2b69600640 1 -- 172.21.15.47:0/3861169687 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c037070 con 0x7f2b6c1514f0 2024-04-24T22:40:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.733+0000 7f2b69600640 1 -- 172.21.15.47:0/3861169687 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b5c032440 con 0x7f2b6c1514f0 2024-04-24T22:40:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.733+0000 7f2b69600640 1 -- 172.21.15.47:0/3861169687 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c032810 con 0x7f2b6c1514f0 2024-04-24T22:40:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.733+0000 7f2b6be00640 1 -- 172.21.15.47:0/3861169687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1514f0 msgr2=0x7f2b6c06e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.733+0000 7f2b6be00640 1 --2- 172.21.15.47:0/3861169687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1514f0 0x7f2b6c06e1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0097a0 tx=0x7f2b5c02a180 comp rx=0 tx=0).stop 2024-04-24T22:40:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.733+0000 7f2b6be00640 1 -- 172.21.15.47:0/3861169687 shutdown_connections 2024-04-24T22:40:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.733+0000 7f2b6be00640 1 --2- 172.21.15.47:0/3861169687 >> v2:172.21.15.113:3300/0 conn(0x7f2b6c0c6a20 0x7f2b6c0707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.733+0000 7f2b6be00640 1 --2- 172.21.15.47:0/3861169687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1514f0 0x7f2b6c06e1b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.733+0000 7f2b6be00640 1 -- 172.21.15.47:0/3861169687 >> 172.21.15.47:0/3861169687 conn(0x7f2b6c0cb900 msgr2=0x7f2b6c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:39.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.734+0000 7f2b6be00640 1 -- 172.21.15.47:0/3861169687 shutdown_connections 2024-04-24T22:40:39.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.734+0000 7f2b6be00640 1 -- 172.21.15.47:0/3861169687 wait complete. 2024-04-24T22:40:39.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.735+0000 7f2b6be00640 1 Processor -- start 2024-04-24T22:40:39.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.735+0000 7f2b6be00640 1 -- start start 2024-04-24T22:40:39.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.736+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c06e6f0 0x7f2b6c1f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.736+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b6c0c6a20 0x7f2b6c1ea410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.736+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b6c1514f0 0x7f2b6c1427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.736+0000 7f2b6be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c15b100 con 0x7f2b6c1514f0 2024-04-24T22:40:39.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.736+0000 7f2b6be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c15ae20 con 0x7f2b6c0c6a20 2024-04-24T22:40:39.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.736+0000 7f2b6be00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c0c7ba0 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6b400640 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b6c1514f0 msgr2=0x7f2b6c1427a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-24T22:40:39.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b6c1514f0 0x7f2b6c1427a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:40:39.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c06e6f0 0x7f2b6c1f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b6c0c6a20 0x7f2b6c1ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c06e6f0 0x7f2b6c1f0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:33822/0 (socket says 172.21.15.47:33822) 2024-04-24T22:40:39.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6aa00640 1 -- 172.21.15.47:0/2279018680 learned_addr learned my addr 172.21.15.47:0/2279018680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:40:39.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6aa00640 1 -- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b6c0c6a20 msgr2=0x7f2b6c1ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6aa00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b6c0c6a20 0x7f2b6c1ea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6aa00640 1 -- 172.21.15.47:0/2279018680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b6c1514f0 msgr2=0x7f2b6c1427a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:40:39.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6aa00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b6c1514f0 0x7f2b6c1427a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.737+0000 7f2b6aa00640 1 -- 172.21.15.47:0/2279018680 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c0c9ae0 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.738+0000 7f2b6aa00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c06e6f0 0x7f2b6c1f0c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0098c0 tx=0x7f2b5c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:39.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.738+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c041070 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.738+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b5c03c870 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.738+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c037040 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.738+0000 7f2b6be00640 1 -- 172.21.15.47:0/2279018680 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c0c9dc0 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.738+0000 7f2b6be00640 1 -- 172.21.15.47:0/2279018680 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c1e9c10 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.739+0000 7f2b6be00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1ea950 0x7f2b6c142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.739+0000 7f2b6be00640 1 -- 172.21.15.47:0/2279018680 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c0ca2e0 con 0x7f2b6c1ea950 2024-04-24T22:40:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.739+0000 7f2b6a000640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1ea950 0x7f2b6c142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.740+0000 7f2b6a000640 1 -- 172.21.15.47:0/2279018680 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b6c1e9a30 con 0x7f2b6c1ea950 2024-04-24T22:40:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.740+0000 7f2b6a000640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1ea950 0x7f2b6c142ce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c073860 tx=0x7f2b5800f9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.740+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b5c044070 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.741+0000 7f2b62a00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2b4c06be10 0x7f2b4c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.741+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f2b5c0ab4d0 con 0x7f2b6c06e6f0 2024-04-24T22:40:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.741+0000 7f2b6b400640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2b4c06be10 0x7f2b4c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.741+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58005ac0 con 0x7f2b6c1ea950 2024-04-24T22:40:39.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.741+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2b58018070 con 0x7f2b6c1ea950 2024-04-24T22:40:39.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.741+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1ea950 msgr2=0x7f2b6c142ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.741+0000 7f2b62a00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1ea950 0x7f2b6c142ce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c073860 tx=0x7f2b5800f9b0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.741+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1ea950 msgr2=0x7f2b6c142ce0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:40:39.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.741+0000 7f2b62a00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1ea950 0x7f2b6c142ce0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c073860 tx=0x7f2b5800f9b0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.742+0000 7f2b6b400640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2b4c06be10 0x7f2b4c06e2d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009bd0 tx=0x7f2b6400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:39.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.749+0000 7f2b6be00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1e9e50 0x7f2b6c159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:39.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.749+0000 7f2b6be00640 1 -- 172.21.15.47:0/2279018680 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c068d50 con 0x7f2b6c1e9e50 2024-04-24T22:40:39.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.749+0000 7f2b6a000640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1e9e50 0x7f2b6c159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:39.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.750+0000 7f2b6a000640 1 -- 172.21.15.47:0/2279018680 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2b6c0c2750 con 0x7f2b6c1e9e50 2024-04-24T22:40:39.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.750+0000 7f2b6a000640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1e9e50 0x7f2b6c159410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2b58014000 tx=0x7f2b580010a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:40:39.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.751+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5801ec60 con 0x7f2b6c1e9e50 2024-04-24T22:40:39.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.751+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5195 (secure 0 0 0) 0x7f2b5801ec60 con 0x7f2b6c1e9e50 2024-04-24T22:40:39.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.751+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1e9e50 msgr2=0x7f2b6c159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.751+0000 7f2b62a00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1e9e50 0x7f2b6c159410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2b58014000 tx=0x7f2b580010a0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.751+0000 7f2b62a00640 1 -- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1e9e50 msgr2=0x7f2b6c159410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:40:39.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.751+0000 7f2b62a00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c1e9e50 0x7f2b6c159410 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.757 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:40:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:40:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:40:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:40:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 6, 2024-04-24T22:40:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 81317, 2024-04-24T22:40:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:40:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:40:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:40:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:40:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:40:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:40:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:40:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:40:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:40:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:40:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:40:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:40:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:40:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:40:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:40:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:40:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:40:39.764 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:40:39.764 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:40:39.764 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:40:39.764 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:40:39.765 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.765 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:40:39.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.765 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:40:39.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.765 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:40:39.765 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:40:39.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:39.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:39.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:39.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:40:39.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:39.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:40:39.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.767 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:40:39.768 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:40:39.768 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:39.768 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:39.768 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:39.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.769 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:40:39.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:40:39.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:39.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:39.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:39.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:40:39.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:39.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:40:39.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.771 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.771 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:40:39.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:40:39.771 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:39.771 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:39.772 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:39.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.773 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:40:39.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:40:39.773 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:40:39.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:40:39.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:40:39.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:40:39.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:40:39.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:40:39.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:40:39.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:40:39.776 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:40:39.776 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:40:39.777 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:40:39.777 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:40:39.777 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.777 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.777 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:40:39.777 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:40:39.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.778 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:39.778 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.778 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:39.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.778 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.779 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:40:39.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.779 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:39.779 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.779 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:40:39.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.779 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:40:39.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.780 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:39.780 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.781 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:40:39.781 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.781 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.781 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:40:39.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.781 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:40:39.781 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.781 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:39.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.782 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:39.782 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.782 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-24T22:40:39.782 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.782 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:40:39.783 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:40:39.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:40:39.784 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:40:39.784 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:40:39.784 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:40:39.784 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:40:39.784 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:40:39.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:40:39.784 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:40:39.784 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:40:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.751+0000 7f2b6be00640 1 -- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2b4c06be10 msgr2=0x7f2b4c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.751+0000 7f2b6be00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2b4c06be10 0x7f2b4c06e2d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009bd0 tx=0x7f2b6400b040 comp rx=0 tx=0).stop 2024-04-24T22:40:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.751+0000 7f2b6be00640 1 -- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c06e6f0 msgr2=0x7f2b6c1f0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:40:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.751+0000 7f2b6be00640 1 --2- 172.21.15.47:0/2279018680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b6c06e6f0 0x7f2b6c1f0c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0098c0 tx=0x7f2b5c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:40:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.752+0000 7f2b6b400640 1 -- 172.21.15.47:0/2279018680 reap_dead start 2024-04-24T22:40:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.752+0000 7f2b6b400640 1 -- 172.21.15.47:0/2279018680 reap_dead start 2024-04-24T22:40:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.752+0000 7f2b6be00640 1 -- 172.21.15.47:0/2279018680 shutdown_connections 2024-04-24T22:40:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.752+0000 7f2b6be00640 1 -- 172.21.15.47:0/2279018680 >> 172.21.15.47:0/2279018680 conn(0x7f2b6c0cb900 msgr2=0x7f2b6c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:40:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.752+0000 7f2b6be00640 1 -- 172.21.15.47:0/2279018680 shutdown_connections 2024-04-24T22:40:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:39.752+0000 7f2b6be00640 1 -- 172.21.15.47:0/2279018680 wait complete. 2024-04-24T22:40:39.789 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-24T22:40:39.789 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-24T22:40:40.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:40:40.050 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:40:59.789 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-24T22:40:59.789 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-04-24T22:40:59.790 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-24T22:40:59.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-24T22:40:59.796 INFO:tasks.ceph.mon.a:Started 2024-04-24T22:40:59.796 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-24T22:40:59.797 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:40:59.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.988+0000 7fccb6e00640 1 Processor -- start 2024-04-24T22:40:59.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.988+0000 7fccb6e00640 1 -- start start 2024-04-24T22:40:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccb6e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fccb00c2e80 0x7fccb00c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:40:59.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccb6e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb015a880 con 0x7fccb00c4250 2024-04-24T22:40:59.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccb6e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb015ab60 con 0x7fccb0157bd0 2024-04-24T22:40:59.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccb6e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fccb00715b0 con 0x7fccb00c2e80 2024-04-24T22:40:59.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccb4e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fccb00c4250 0x7fccb015bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48176/0 (socket says 172.21.15.47:48176) 2024-04-24T22:40:59.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccb4e00640 1 -- 172.21.15.47:0/3008501335 learned_addr learned my addr 172.21.15.47:0/3008501335 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:40:59.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccaf400640 1 --2- 172.21.15.47:0/3008501335 >> v2:172.21.15.113:3300/0 conn(0x7fccb00c2e80 0x7fccb00c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:40:59.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccaf400640 1 -- 172.21.15.47:0/3008501335 >> v2:172.21.15.113:3300/0 conn(0x7fccb00c2e80 msgr2=0x7fccb00c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:40:59.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccaf400640 1 -- 172.21.15.47:0/3008501335 >> v2:172.21.15.113:3300/0 conn(0x7fccb00c2e80 msgr2=0x7fccb00c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:40:59.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccaf400640 1 --2- 172.21.15.47:0/3008501335 >> v2:172.21.15.113:3300/0 conn(0x7fccb00c2e80 0x7fccb00c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:40:59.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:40:59.990+0000 7fccaf400640 1 --2- 172.21.15.47:0/3008501335 >> v2:172.21.15.113:3300/0 conn(0x7fccb00c2e80 0x7fccb00c8760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.181+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761606139 0 0) 0x7fccb015ab60 con 0x7fccb0157bd0 2024-04-24T22:41:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.181+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc94003520 con 0x7fccb0157bd0 2024-04-24T22:41:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.182+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 155789244 0 0) 0x7fcc94003520 con 0x7fccb0157bd0 2024-04-24T22:41:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.182+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 >> v2:172.21.15.113:3300/0 conn(0x7fccb00c2e80 msgr2=0x7fccb00c8760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:41:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.182+0000 7fccaea00640 1 --2- 172.21.15.47:0/3008501335 >> v2:172.21.15.113:3300/0 conn(0x7fccb00c2e80 0x7fccb00c8760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.182+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 >> v1:172.21.15.47:6789/0 conn(0x7fccb00c4250 legacy=0x7fccb015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.182+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb015ae40 con 0x7fccb0157bd0 2024-04-24T22:41:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.182+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 567141988 0 0) 0x7fcca00015a0 con 0x7fccb0157bd0 2024-04-24T22:41:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.182+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 >> v1:172.21.15.113:6789/0 conn(0x7fccb0157bd0 legacy=0x7fccb0072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.183+0000 7fccaea00640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94003a00 0x7fcc94003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.183+0000 7fccaea00640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 0x7fcc94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.183+0000 7fccaea00640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc94005210 0x7fcc9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.183+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9400c120 con 0x7fcc94003a00 2024-04-24T22:41:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.183+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9400c2a0 con 0x7fcc94005210 2024-04-24T22:41:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.183+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcc9400c4a0 con 0x7fcc94004800 2024-04-24T22:41:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.183+0000 7fccaf400640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 0x7fcc94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.183+0000 7fccb4e00640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc94005210 0x7fcc9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.183+0000 7fccafe00640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94003a00 0x7fcc94003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.184+0000 7fccaf400640 1 -- 172.21.15.47:0/3008501335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc94005210 msgr2=0x7fcc9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.184+0000 7fccaf400640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc94005210 0x7fcc9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.184+0000 7fccaf400640 1 -- 172.21.15.47:0/3008501335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94003a00 msgr2=0x7fcc94003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.184+0000 7fccaf400640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94003a00 0x7fcc94003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.184+0000 7fccaf400640 1 -- 172.21.15.47:0/3008501335 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc94003520 con 0x7fcc94004800 2024-04-24T22:41:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.184+0000 7fccb4e00640 1 -- 172.21.15.47:0/3008501335 reap_dead start 2024-04-24T22:41:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.184+0000 7fccafe00640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94003a00 0x7fcc94003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.186+0000 7fccaf400640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 0x7fcc94004c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00b9e0 tx=0x7fcc94005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cd5528ffee9ecda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.187+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c030070 con 0x7fcc94004800 2024-04-24T22:41:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.187+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c02ad70 con 0x7fcc94004800 2024-04-24T22:41:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.187+0000 7fccaea00640 1 -- 172.21.15.47:0/3008501335 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00b5e0 con 0x7fcc94004800 2024-04-24T22:41:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.188+0000 7fccb6e00640 1 -- 172.21.15.47:0/3008501335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 msgr2=0x7fcc94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.188+0000 7fccb6e00640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 0x7fcc94004c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00b9e0 tx=0x7fcc94005830 comp rx=0 tx=0).stop 2024-04-24T22:41:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.189+0000 7fccb6e00640 1 -- 172.21.15.47:0/3008501335 shutdown_connections 2024-04-24T22:41:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.189+0000 7fccb6e00640 1 --2- 172.21.15.47:0/3008501335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 0x7fcc94004c80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.189+0000 7fccb6e00640 1 -- 172.21.15.47:0/3008501335 >> 172.21.15.47:0/3008501335 conn(0x7fccb006e730 msgr2=0x7fccb006eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:00.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.189+0000 7fccb6e00640 1 -- 172.21.15.47:0/3008501335 shutdown_connections 2024-04-24T22:41:00.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.189+0000 7fccb6e00640 1 -- 172.21.15.47:0/3008501335 wait complete. 2024-04-24T22:41:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.196+0000 7fccb6e00640 1 Processor -- start 2024-04-24T22:41:00.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.197+0000 7fccb6e00640 1 -- start start 2024-04-24T22:41:00.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.198+0000 7fccb6e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 0x7fccb00c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccb6e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb0158750 0x7fccb0158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccb6e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb014fcc0 0x7fccb0150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccb6e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fccb015b490 con 0x7fccb014fcc0 2024-04-24T22:41:00.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccb6e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fccb00c8e60 con 0x7fccb0158750 2024-04-24T22:41:00.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccb6e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fccb0157fb0 con 0x7fcc94004800 2024-04-24T22:41:00.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccafe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 0x7fccb00c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccaf400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb0158750 0x7fccb0158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccafe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 0x7fccb00c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:60946/0 (socket says 172.21.15.47:60946) 2024-04-24T22:41:00.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccafe00640 1 -- 172.21.15.47:0/1099554627 learned_addr learned my addr 172.21.15.47:0/1099554627 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:00.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccb4e00640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb014fcc0 0x7fccb0150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccaf400640 1 -- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 msgr2=0x7fccb00c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccaf400640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 0x7fccb00c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccaf400640 1 -- 172.21.15.47:0/1099554627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb014fcc0 msgr2=0x7fccb0150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccaf400640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb014fcc0 0x7fccb0150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.199+0000 7fccaf400640 1 -- 172.21.15.47:0/1099554627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb0155cd0 con 0x7fccb0158750 2024-04-24T22:41:00.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.200+0000 7fccaf400640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb0158750 0x7fccb0158bb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400c670 tx=0x7fcc9c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.200+0000 7fcca7e00640 1 -- 172.21.15.47:0/1099554627 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c042020 con 0x7fccb0158750 2024-04-24T22:41:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.200+0000 7fcca7e00640 1 -- 172.21.15.47:0/1099554627 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c03d650 con 0x7fccb0158750 2024-04-24T22:41:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.200+0000 7fcca7e00640 1 -- 172.21.15.47:0/1099554627 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00b5e0 con 0x7fccb0158750 2024-04-24T22:41:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.200+0000 7fccb6e00640 1 -- 172.21.15.47:0/1099554627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb01590f0 con 0x7fccb0158750 2024-04-24T22:41:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.200+0000 7fccb6e00640 1 -- 172.21.15.47:0/1099554627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb01f04b0 con 0x7fccb0158750 2024-04-24T22:41:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.201+0000 7fccb6e00640 1 -- 172.21.15.47:0/1099554627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb0073b90 con 0x7fccb0158750 2024-04-24T22:41:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.202+0000 7fcca7e00640 1 -- 172.21.15.47:0/1099554627 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc9c045020 con 0x7fccb0158750 2024-04-24T22:41:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.203+0000 7fcca7e00640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcc9006be60 0x7fcc9006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.203+0000 7fcca7e00640 1 -- 172.21.15.47:0/1099554627 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcc9c0ab500 con 0x7fccb0158750 2024-04-24T22:41:00.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.204+0000 7fccafe00640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcc9006be60 0x7fcc9006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.204+0000 7fccafe00640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcc9006be60 0x7fcc9006e320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcca0001b90 tx=0x7fcca0001b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.208+0000 7fcca7e00640 1 -- 172.21.15.47:0/1099554627 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcc9c04c030 con 0x7fccb0158750 2024-04-24T22:41:00.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.495+0000 7fccb6e00640 1 -- 172.21.15.47:0/1099554627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fccb015dad0 con 0x7fccb0158750 2024-04-24T22:41:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.496+0000 7fcca7e00640 1 -- 172.21.15.47:0/1099554627 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7fcc9c079750 con 0x7fccb0158750 2024-04-24T22:41:00.498 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:41:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.498+0000 7fccb6e00640 1 -- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcc9006be60 msgr2=0x7fcc9006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.498+0000 7fccb6e00640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcc9006be60 0x7fcc9006e320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcca0001b90 tx=0x7fcca0001b20 comp rx=0 tx=0).stop 2024-04-24T22:41:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.498+0000 7fccb6e00640 1 -- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb0158750 msgr2=0x7fccb0158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.498+0000 7fccb6e00640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb0158750 0x7fccb0158bb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400c670 tx=0x7fcc9c004730 comp rx=0 tx=0).stop 2024-04-24T22:41:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.498+0000 7fccb6e00640 1 -- 172.21.15.47:0/1099554627 shutdown_connections 2024-04-24T22:41:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.498+0000 7fccb6e00640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcc9006be60 0x7fcc9006e320 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.498+0000 7fccb6e00640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb014fcc0 0x7fccb0150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.499+0000 7fccb6e00640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb0158750 0x7fccb0158bb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.499+0000 7fccb6e00640 1 --2- 172.21.15.47:0/1099554627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc94004800 0x7fccb00c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.499+0000 7fccb6e00640 1 -- 172.21.15.47:0/1099554627 >> 172.21.15.47:0/1099554627 conn(0x7fccb006e730 msgr2=0x7fccb00c5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.499+0000 7fccb6e00640 1 -- 172.21.15.47:0/1099554627 shutdown_connections 2024-04-24T22:41:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.499+0000 7fccb6e00640 1 -- 172.21.15.47:0/1099554627 wait complete. 2024-04-24T22:41:00.520 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-24T22:41:00.520 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-24T22:41:00.521 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:41:00.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.720+0000 7f7ff3e00640 1 Processor -- start 2024-04-24T22:41:00.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.720+0000 7f7ff3e00640 1 -- start start 2024-04-24T22:41:00.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.722+0000 7f7ff3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7ff40c5320 0x7f7ff40c2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.722+0000 7f7ff3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ff406b160 0x7f7ff406b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.722+0000 7f7ff3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff40c28d0 con 0x7f7ff4157420 2024-04-24T22:41:00.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.722+0000 7f7ff3e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff4070ef0 con 0x7f7ff406b160 2024-04-24T22:41:00.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.722+0000 7f7ff3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff4151160 con 0x7f7ff40c5320 2024-04-24T22:41:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.723+0000 7f7ff2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7ff40c5320 0x7f7ff40c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.723+0000 7f7ff3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ff406b160 0x7f7ff406b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.723+0000 7f7febe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7ff4157420 0x7f7ff40707e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:56744/0 (socket says 172.21.15.47:56744) 2024-04-24T22:41:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.723+0000 7f7febe00640 1 -- 172.21.15.47:0/2947808153 learned_addr learned my addr 172.21.15.47:0/2947808153 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.723+0000 7f7ff2a00640 1 -- 172.21.15.47:0/2947808153 >> v2:172.21.15.47:3300/0 conn(0x7f7ff406b160 msgr2=0x7f7ff406b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.723+0000 7f7ff2a00640 1 --2- 172.21.15.47:0/2947808153 >> v2:172.21.15.47:3300/0 conn(0x7f7ff406b160 0x7f7ff406b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.723+0000 7f7ff2a00640 1 -- 172.21.15.47:0/2947808153 >> v1:172.21.15.113:6789/0 conn(0x7f7ff4157420 legacy=0x7f7ff40707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.723+0000 7f7ff2a00640 1 -- 172.21.15.47:0/2947808153 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff4072080 con 0x7f7ff40c5320 2024-04-24T22:41:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.724+0000 7f7ff2a00640 1 --2- 172.21.15.47:0/2947808153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ff40c5320 0x7f7ff40c2290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ff41512e0 tx=0x7f7fe402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e459e25500d0a7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.724+0000 7f7ff2000640 1 -- 172.21.15.47:0/2947808153 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4037070 con 0x7f7ff40c5320 2024-04-24T22:41:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.725+0000 7f7ff2000640 1 -- 172.21.15.47:0/2947808153 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe402adc0 con 0x7f7ff40c5320 2024-04-24T22:41:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.725+0000 7f7ff2000640 1 -- 172.21.15.47:0/2947808153 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4032660 con 0x7f7ff40c5320 2024-04-24T22:41:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.725+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2947808153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ff40c5320 msgr2=0x7f7ff40c2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.725+0000 7f7ff3e00640 1 --2- 172.21.15.47:0/2947808153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ff40c5320 0x7f7ff40c2290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ff41512e0 tx=0x7f7fe402a130 comp rx=0 tx=0).stop 2024-04-24T22:41:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.725+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2947808153 shutdown_connections 2024-04-24T22:41:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.725+0000 7f7ff3e00640 1 --2- 172.21.15.47:0/2947808153 >> v2:172.21.15.47:3300/0 conn(0x7f7ff406b160 0x7f7ff406b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.725+0000 7f7ff3e00640 1 --2- 172.21.15.47:0/2947808153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ff40c5320 0x7f7ff40c2290 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.725+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2947808153 >> 172.21.15.47:0/2947808153 conn(0x7f7ff40cb900 msgr2=0x7f7ff40cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.725+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2947808153 shutdown_connections 2024-04-24T22:41:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.726+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2947808153 wait complete. 2024-04-24T22:41:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.727+0000 7f7ff3e00640 1 Processor -- start 2024-04-24T22:41:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.728+0000 7f7ff3e00640 1 -- start start 2024-04-24T22:41:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7ff3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ff406b160 0x7f7ff4150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7ff3e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ff40c5320 0x7f7ff41529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7ff3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff4157420 0x7f7ff4152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7ff3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff4157840 con 0x7f7ff4157420 2024-04-24T22:41:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7ff3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff4072640 con 0x7f7ff406b160 2024-04-24T22:41:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7ff3e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff4072e40 con 0x7f7ff40c5320 2024-04-24T22:41:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7ff3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff4157420 0x7f7ff4152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7febe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ff40c5320 0x7f7ff41529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7ff3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff4157420 0x7f7ff4152f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36506/0 (socket says 172.21.15.47:36506) 2024-04-24T22:41:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7ff3400640 1 -- 172.21.15.47:0/2564717209 learned_addr learned my addr 172.21.15.47:0/2564717209 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.729+0000 7f7ff2a00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ff406b160 0x7f7ff4150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7febe00640 1 -- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ff406b160 msgr2=0x7f7ff4150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7febe00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ff406b160 0x7f7ff4150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7febe00640 1 -- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff4157420 msgr2=0x7f7ff4152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7febe00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff4157420 0x7f7ff4152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7febe00640 1 -- 172.21.15.47:0/2564717209 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff41dcc00 con 0x7f7ff40c5320 2024-04-24T22:41:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7ff3400640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff4157420 0x7f7ff4152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7febe00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ff40c5320 0x7f7ff41529c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c520 tx=0x7f7fdc00c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc017070 con 0x7f7ff40c5320 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fdc0074e0 con 0x7f7ff40c5320 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc004a20 con 0x7f7ff40c5320 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.730+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2564717209 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff41f0380 con 0x7f7ff40c5320 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.731+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2564717209 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff41f0940 con 0x7f7ff40c5320 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.731+0000 7f7ff3e00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff41524a0 0x7f7ff41f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.731+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2564717209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff4068d50 con 0x7f7ff41524a0 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.732+0000 7f7ff2a00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff41524a0 0x7f7ff41f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.732+0000 7f7ff2a00640 1 -- 172.21.15.47:0/2564717209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ff404f5e0 con 0x7f7ff41524a0 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.733+0000 7f7ff2a00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff41524a0 0x7f7ff41f0b80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40cc910 tx=0x7f7fe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.733+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4037040 con 0x7f7ff41524a0 2024-04-24T22:41:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.733+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fdc01a070 con 0x7f7ff40c5320 2024-04-24T22:41:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.734+0000 7f7feaa00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7fd006bd60 0x7f7fd006e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.734+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f7fdc08a250 con 0x7f7ff40c5320 2024-04-24T22:41:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.734+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7fe4003810 con 0x7f7ff41524a0 2024-04-24T22:41:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.734+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff41524a0 msgr2=0x7f7ff41f0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.734+0000 7f7feaa00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff41524a0 0x7f7ff41f0b80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40cc910 tx=0x7f7fe40046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.734+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff41524a0 msgr2=0x7f7ff41f0b80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.734+0000 7f7feaa00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff41524a0 0x7f7ff41f0b80 secure :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40cc910 tx=0x7f7fe40046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.734+0000 7f7ff3400640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7fd006bd60 0x7f7fd006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.735+0000 7f7ff3400640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7fd006bd60 0x7f7fd006e220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0094c0 tx=0x7f7fec00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.742+0000 7f7ff3e00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff406eb20 0x7f7ff414e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.742+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2564717209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff4070a30 con 0x7f7ff406eb20 2024-04-24T22:41:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.742+0000 7f7ff2a00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff406eb20 0x7f7ff414e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.742+0000 7f7ff2a00640 1 -- 172.21.15.47:0/2564717209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7ff41e2eb0 con 0x7f7ff406eb20 2024-04-24T22:41:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7ff2a00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff406eb20 0x7f7ff414e2c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4033000 tx=0x7f7fe4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe403d640 con 0x7f7ff406eb20 2024-04-24T22:41:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4554 (secure 0 0 0) 0x7f7fe403d640 con 0x7f7ff406eb20 2024-04-24T22:41:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff406eb20 msgr2=0x7f7ff414e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7feaa00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff406eb20 0x7f7ff414e2c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4033000 tx=0x7f7fe4009190 comp rx=0 tx=0).stop 2024-04-24T22:41:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7feaa00640 1 -- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff406eb20 msgr2=0x7f7ff414e2c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7feaa00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ff406eb20 0x7f7ff414e2c0 secure :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4033000 tx=0x7f7fe4009190 comp rx=0 tx=0).stop 2024-04-24T22:41:00.746 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:00.746 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 10, 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 780, 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:00.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:00.749 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:00.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:00.751 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.754 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:00.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:00.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:00.757 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7fd006bd60 msgr2=0x7f7fd006e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7ff3e00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7fd006bd60 0x7f7fd006e220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0094c0 tx=0x7f7fec00b040 comp rx=0 tx=0).stop 2024-04-24T22:41:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ff40c5320 msgr2=0x7f7ff41529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7ff3e00640 1 --2- 172.21.15.47:0/2564717209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ff40c5320 0x7f7ff41529c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c520 tx=0x7f7fdc00c9e0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.743+0000 7f7ff3400640 1 -- 172.21.15.47:0/2564717209 reap_dead start 2024-04-24T22:41:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.744+0000 7f7ff3400640 1 -- 172.21.15.47:0/2564717209 reap_dead start 2024-04-24T22:41:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.744+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2564717209 shutdown_connections 2024-04-24T22:41:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.744+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2564717209 >> 172.21.15.47:0/2564717209 conn(0x7f7ff40cb900 msgr2=0x7f7ff40c6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.744+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2564717209 shutdown_connections 2024-04-24T22:41:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.744+0000 7f7ff3e00640 1 -- 172.21.15.47:0/2564717209 wait complete. 2024-04-24T22:41:00.762 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:41:00.762 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-24T22:41:00.762 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:41:00.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.968+0000 7fbe12800640 1 Processor -- start 2024-04-24T22:41:00.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.968+0000 7fbe12800640 1 -- start start 2024-04-24T22:41:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.969+0000 7fbe12800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe0c1522d0 0x7fbe0c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.969+0000 7fbe12800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe0c0c85e0 0x7fbe0c15b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.969+0000 7fbe12800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe0c1527f0 con 0x7fbe0c06bee0 2024-04-24T22:41:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.969+0000 7fbe12800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbe0c158810 con 0x7fbe0c1522d0 2024-04-24T22:41:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.969+0000 7fbe12800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbe0c0c8b40 con 0x7fbe0c0c85e0 2024-04-24T22:41:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.969+0000 7fbe11400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe0c1522d0 0x7fbe0c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.970+0000 7fbe11400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe0c1522d0 0x7fbe0c158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36524/0 (socket says 172.21.15.47:36524) 2024-04-24T22:41:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.970+0000 7fbe11400640 1 -- 172.21.15.47:0/1343770319 learned_addr learned my addr 172.21.15.47:0/1343770319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.970+0000 7fbe10a00640 1 --2- 172.21.15.47:0/1343770319 >> v2:172.21.15.113:3300/0 conn(0x7fbe0c0c85e0 0x7fbe0c15b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.970+0000 7fbe10a00640 1 -- 172.21.15.47:0/1343770319 >> v2:172.21.15.47:3300/0 conn(0x7fbe0c1522d0 msgr2=0x7fbe0c158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:00.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.970+0000 7fbe10a00640 1 --2- 172.21.15.47:0/1343770319 >> v2:172.21.15.47:3300/0 conn(0x7fbe0c1522d0 0x7fbe0c158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.970+0000 7fbe10a00640 1 -- 172.21.15.47:0/1343770319 >> v1:172.21.15.47:6789/0 conn(0x7fbe0c06bee0 legacy=0x7fbe0c06b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.970+0000 7fbe10a00640 1 -- 172.21.15.47:0/1343770319 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c0c8e20 con 0x7fbe0c0c85e0 2024-04-24T22:41:00.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.970+0000 7fbe11400640 1 --2- 172.21.15.47:0/1343770319 >> v2:172.21.15.47:3300/0 conn(0x7fbe0c1522d0 0x7fbe0c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:00.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.971+0000 7fbe10a00640 1 --2- 172.21.15.47:0/1343770319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c85e0 0x7fbe0c15b740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c158990 tx=0x7fbdf802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5595bb615c2c6d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe0be00640 1 -- 172.21.15.47:0/1343770319 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8037070 con 0x7fbe0c0c85e0 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe0be00640 1 -- 172.21.15.47:0/1343770319 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf8032440 con 0x7fbe0c0c85e0 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe0be00640 1 -- 172.21.15.47:0/1343770319 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8032810 con 0x7fbe0c0c85e0 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe12800640 1 -- 172.21.15.47:0/1343770319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c85e0 msgr2=0x7fbe0c15b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe12800640 1 --2- 172.21.15.47:0/1343770319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c85e0 0x7fbe0c15b740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c158990 tx=0x7fbdf802a230 comp rx=0 tx=0).stop 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe12800640 1 -- 172.21.15.47:0/1343770319 shutdown_connections 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe12800640 1 --2- 172.21.15.47:0/1343770319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c85e0 0x7fbe0c15b740 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe12800640 1 --2- 172.21.15.47:0/1343770319 >> v2:172.21.15.47:3300/0 conn(0x7fbe0c1522d0 0x7fbe0c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe12800640 1 -- 172.21.15.47:0/1343770319 >> 172.21.15.47:0/1343770319 conn(0x7fbe0c0c9410 msgr2=0x7fbe0c0c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe12800640 1 -- 172.21.15.47:0/1343770319 shutdown_connections 2024-04-24T22:41:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.972+0000 7fbe12800640 1 -- 172.21.15.47:0/1343770319 wait complete. 2024-04-24T22:41:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.975+0000 7fbe12800640 1 Processor -- start 2024-04-24T22:41:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.975+0000 7fbe12800640 1 -- start start 2024-04-24T22:41:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe12800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c06bee0 0x7fbe0c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe12800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe0c0c85e0 0x7fbe0c1e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe12800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe0c1522d0 0x7fbe0c1e17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe12800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c158150 con 0x7fbe0c1522d0 2024-04-24T22:41:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe12800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1584a0 con 0x7fbe0c06bee0 2024-04-24T22:41:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe12800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0c06b870 con 0x7fbe0c0c85e0 2024-04-24T22:41:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe11e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe0c1522d0 0x7fbe0c1e17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe11e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe0c1522d0 0x7fbe0c1e17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36536/0 (socket says 172.21.15.47:36536) 2024-04-24T22:41:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe11e00640 1 -- 172.21.15.47:0/714912202 learned_addr learned my addr 172.21.15.47:0/714912202 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe11400640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c06bee0 0x7fbe0c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.977+0000 7fbe10a00640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe0c0c85e0 0x7fbe0c1e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe11e00640 1 -- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe0c0c85e0 msgr2=0x7fbe0c1e43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe11e00640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe0c0c85e0 0x7fbe0c1e43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe11e00640 1 -- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c06bee0 msgr2=0x7fbe0c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe11e00640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c06bee0 0x7fbe0c0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe11e00640 1 -- 172.21.15.47:0/714912202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c0ca9c0 con 0x7fbe0c1522d0 2024-04-24T22:41:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe11e00640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe0c1522d0 0x7fbe0c1e17d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400c7d0 tx=0x7fbe0400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04003b70 con 0x7fbe0c1522d0 2024-04-24T22:41:00.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe04003d30 con 0x7fbe0c1522d0 2024-04-24T22:41:00.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0400aa90 con 0x7fbe0c1522d0 2024-04-24T22:41:00.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe12800640 1 -- 172.21.15.47:0/714912202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c0caca0 con 0x7fbe0c1522d0 2024-04-24T22:41:00.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.978+0000 7fbe12800640 1 -- 172.21.15.47:0/714912202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c1e2420 con 0x7fbe0c1522d0 2024-04-24T22:41:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.979+0000 7fbe12800640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c2d40 0x7fbe0c1e28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.979+0000 7fbe12800640 1 -- 172.21.15.47:0/714912202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1e4900 con 0x7fbe0c0c2d40 2024-04-24T22:41:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.980+0000 7fbe11400640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c2d40 0x7fbe0c1e28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.980+0000 7fbe11400640 1 -- 172.21.15.47:0/714912202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe0c0cc6b0 con 0x7fbe0c0c2d40 2024-04-24T22:41:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.980+0000 7fbe11400640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c2d40 0x7fbe0c1e28d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe00011840 tx=0x7fbe00011d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.980+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000faa0 con 0x7fbe0c0c2d40 2024-04-24T22:41:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.981+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbe0000fc40 con 0x7fbe0c0c2d40 2024-04-24T22:41:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.981+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c2d40 msgr2=0x7fbe0c1e28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.981+0000 7fbe09600640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c2d40 0x7fbe0c1e28d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe00011840 tx=0x7fbe00011d00 comp rx=0 tx=0).stop 2024-04-24T22:41:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.981+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c2d40 msgr2=0x7fbe0c1e28d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.981+0000 7fbe09600640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c0c2d40 0x7fbe0c1e28d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.982+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe0400ac30 con 0x7fbe0c1522d0 2024-04-24T22:41:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.983+0000 7fbe09600640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbdec06b9c0 0x7fbdec06de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.983+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbe0408ac20 con 0x7fbe0c1522d0 2024-04-24T22:41:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.983+0000 7fbe11400640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbdec06b9c0 0x7fbdec06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.984+0000 7fbe11400640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbdec06b9c0 0x7fbdec06de80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000f6b0 tx=0x7fbe00008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.988+0000 7fbe12800640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c14a280 0x7fbe0c1e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:00.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.988+0000 7fbe12800640 1 -- 172.21.15.47:0/714912202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1e3d00 con 0x7fbe0c14a280 2024-04-24T22:41:00.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.989+0000 7fbe10a00640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c14a280 0x7fbe0c1e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:00.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.989+0000 7fbe10a00640 1 -- 172.21.15.47:0/714912202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbe0c1e3ed0 con 0x7fbe0c14a280 2024-04-24T22:41:00.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.990+0000 7fbe10a00640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c14a280 0x7fbe0c1e2e10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0cc940 tx=0x7fbdf80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:00.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.990+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8043070 con 0x7fbe0c14a280 2024-04-24T22:41:00.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.990+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5065 (secure 0 0 0) 0x7fbdf80093c0 con 0x7fbe0c14a280 2024-04-24T22:41:00.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.990+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c14a280 msgr2=0x7fbe0c1e2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:00.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.990+0000 7fbe09600640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c14a280 0x7fbe0c1e2e10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0cc940 tx=0x7fbdf80046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.990+0000 7fbe09600640 1 -- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c14a280 msgr2=0x7fbe0c1e2e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:00.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.990+0000 7fbe09600640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe0c14a280 0x7fbe0c1e2e10 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0cc940 tx=0x7fbdf80046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:00.993 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:00.993 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:00.993 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:00.993 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:41:00.993 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 10, 2024-04-24T22:41:00.993 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 102558, 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:00.994 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:00.995 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:00.996 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:00.997 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:00.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:00.999 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:01.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:01.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:01.002 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.003 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.004 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:01.005 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:01.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.991+0000 7fbe12800640 1 -- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbdec06b9c0 msgr2=0x7fbdec06de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.991+0000 7fbe12800640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbdec06b9c0 0x7fbdec06de80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000f6b0 tx=0x7fbe00008030 comp rx=0 tx=0).stop 2024-04-24T22:41:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.991+0000 7fbe12800640 1 -- 172.21.15.47:0/714912202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe0c1522d0 msgr2=0x7fbe0c1e17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.991+0000 7fbe12800640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe0c1522d0 0x7fbe0c1e17d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400c7d0 tx=0x7fbe0400cc90 comp rx=0 tx=0).stop 2024-04-24T22:41:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.991+0000 7fbe11e00640 1 -- 172.21.15.47:0/714912202 reap_dead start 2024-04-24T22:41:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.991+0000 7fbe12800640 1 -- 172.21.15.47:0/714912202 shutdown_connections 2024-04-24T22:41:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.991+0000 7fbe12800640 1 --2- 172.21.15.47:0/714912202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe0c1522d0 0x7fbe0c1e17d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.991+0000 7fbe12800640 1 -- 172.21.15.47:0/714912202 >> 172.21.15.47:0/714912202 conn(0x7fbe0c0c9410 msgr2=0x7fbe0c155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.991+0000 7fbe12800640 1 -- 172.21.15.47:0/714912202 shutdown_connections 2024-04-24T22:41:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:00.991+0000 7fbe12800640 1 -- 172.21.15.47:0/714912202 wait complete. 2024-04-24T22:41:01.017 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:41:01.018 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-24T22:41:01.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-24T22:41:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.209+0000 7fb68d000640 1 Processor -- start 2024-04-24T22:41:01.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.210+0000 7fb68d000640 1 -- start start 2024-04-24T22:41:01.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.211+0000 7fb68d000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c8fa0 0x7fb688163dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.211+0000 7fb68d000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6880c7bb0 con 0x7fb6880c7750 2024-04-24T22:41:01.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.211+0000 7fb68d000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6880c9410 con 0x7fb688152960 2024-04-24T22:41:01.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.211+0000 7fb68d000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb688158f20 con 0x7fb6880c8fa0 2024-04-24T22:41:01.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.212+0000 7fb685400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb6880c7750 0x7fb688072b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48188/0 (socket says 172.21.15.47:48188) 2024-04-24T22:41:01.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.212+0000 7fb685400640 1 -- 172.21.15.47:0/3310899366 learned_addr learned my addr 172.21.15.47:0/3310899366 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:01.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.212+0000 7fb686800640 1 --2- 172.21.15.47:0/3310899366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c8fa0 0x7fb688163dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.212+0000 7fb684a00640 1 -- 172.21.15.47:0/3310899366 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1000264243 0 0) 0x7fb6880c9410 con 0x7fb688152960 2024-04-24T22:41:01.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.212+0000 7fb684a00640 1 -- 172.21.15.47:0/3310899366 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb66c003520 con 0x7fb688152960 2024-04-24T22:41:01.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.212+0000 7fb684a00640 1 -- 172.21.15.47:0/3310899366 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3677723672 0 0) 0x7fb6880c7bb0 con 0x7fb6880c7750 2024-04-24T22:41:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.212+0000 7fb684a00640 1 -- 172.21.15.47:0/3310899366 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6880c9410 con 0x7fb6880c7750 2024-04-24T22:41:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.212+0000 7fb686800640 1 -- 172.21.15.47:0/3310899366 >> v1:172.21.15.113:6789/0 conn(0x7fb688152960 legacy=0x7fb68815ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.212+0000 7fb686800640 1 -- 172.21.15.47:0/3310899366 >> v1:172.21.15.47:6789/0 conn(0x7fb6880c7750 legacy=0x7fb688072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.212+0000 7fb686800640 1 -- 172.21.15.47:0/3310899366 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6880c7e50 con 0x7fb6880c8fa0 2024-04-24T22:41:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.213+0000 7fb686800640 1 --2- 172.21.15.47:0/3310899366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c8fa0 0x7fb688163dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb680009dc0 tx=0x7fb68002a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c19e62b60483d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.214+0000 7fb684a00640 1 -- 172.21.15.47:0/3310899366 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680007740 con 0x7fb6880c8fa0 2024-04-24T22:41:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.214+0000 7fb684a00640 1 -- 172.21.15.47:0/3310899366 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6800078e0 con 0x7fb6880c8fa0 2024-04-24T22:41:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.214+0000 7fb684a00640 1 -- 172.21.15.47:0/3310899366 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680007cb0 con 0x7fb6880c8fa0 2024-04-24T22:41:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.214+0000 7fb68d000640 1 -- 172.21.15.47:0/3310899366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c8fa0 msgr2=0x7fb688163dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.214+0000 7fb68d000640 1 --2- 172.21.15.47:0/3310899366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c8fa0 0x7fb688163dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb680009dc0 tx=0x7fb68002a760 comp rx=0 tx=0).stop 2024-04-24T22:41:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.214+0000 7fb68d000640 1 -- 172.21.15.47:0/3310899366 shutdown_connections 2024-04-24T22:41:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.214+0000 7fb68d000640 1 --2- 172.21.15.47:0/3310899366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c8fa0 0x7fb688163dc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.215+0000 7fb68d000640 1 -- 172.21.15.47:0/3310899366 >> 172.21.15.47:0/3310899366 conn(0x7fb68806e580 msgr2=0x7fb68806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:01.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.215+0000 7fb68d000640 1 -- 172.21.15.47:0/3310899366 shutdown_connections 2024-04-24T22:41:01.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.215+0000 7fb68d000640 1 -- 172.21.15.47:0/3310899366 wait complete. 2024-04-24T22:41:01.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.216+0000 7fb68d000640 1 Processor -- start 2024-04-24T22:41:01.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.217+0000 7fb68d000640 1 -- start start 2024-04-24T22:41:01.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb68d000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c7750 0x7fb688153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb68d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880c8fa0 0x7fb68814d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb68d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb688152960 0x7fb6881f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb68d000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb6880c8130 con 0x7fb688152960 2024-04-24T22:41:01.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb68d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881593b0 con 0x7fb6880c8fa0 2024-04-24T22:41:01.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb68d000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb6881588d0 con 0x7fb6880c7750 2024-04-24T22:41:01.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb686800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb688152960 0x7fb6881f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb685e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c7750 0x7fb688153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb685400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880c8fa0 0x7fb68814d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb686800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb688152960 0x7fb6881f7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36546/0 (socket says 172.21.15.47:36546) 2024-04-24T22:41:01.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb686800640 1 -- 172.21.15.47:0/3485981709 learned_addr learned my addr 172.21.15.47:0/3485981709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:01.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb685e00640 1 -- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880c8fa0 msgr2=0x7fb68814d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb685e00640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880c8fa0 0x7fb68814d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb685e00640 1 -- 172.21.15.47:0/3485981709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb688152960 msgr2=0x7fb6881f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.218+0000 7fb685e00640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb688152960 0x7fb6881f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.219+0000 7fb685e00640 1 -- 172.21.15.47:0/3485981709 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6881542f0 con 0x7fb6880c7750 2024-04-24T22:41:01.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.219+0000 7fb685e00640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c7750 0x7fb688153db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c3a0 tx=0x7fb67800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:01.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.219+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678017070 con 0x7fb6880c7750 2024-04-24T22:41:01.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.219+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6780052f0 con 0x7fb6880c7750 2024-04-24T22:41:01.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.219+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678003830 con 0x7fb6880c7750 2024-04-24T22:41:01.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.219+0000 7fb68d000640 1 -- 172.21.15.47:0/3485981709 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6881f89d0 con 0x7fb6880c7750 2024-04-24T22:41:01.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.219+0000 7fb68d000640 1 -- 172.21.15.47:0/3485981709 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6881f86f0 con 0x7fb6880c7750 2024-04-24T22:41:01.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.220+0000 7fb68d000640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880719c0 0x7fb6881f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.220+0000 7fb68d000640 1 -- 172.21.15.47:0/3485981709 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb688073b90 con 0x7fb6880719c0 2024-04-24T22:41:01.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.220+0000 7fb685400640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880719c0 0x7fb6881f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.221+0000 7fb685400640 1 -- 172.21.15.47:0/3485981709 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb68806c200 con 0x7fb6880719c0 2024-04-24T22:41:01.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.221+0000 7fb685400640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880719c0 0x7fb6881f5100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb6881579d0 tx=0x7fb67400f9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:01.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.222+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674005ac0 con 0x7fb6880719c0 2024-04-24T22:41:01.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.222+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb674018070 con 0x7fb6880719c0 2024-04-24T22:41:01.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.222+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880719c0 msgr2=0x7fb6881f5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.222+0000 7fb67e000640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880719c0 0x7fb6881f5100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb6881579d0 tx=0x7fb67400f9d0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.222+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880719c0 msgr2=0x7fb6881f5100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:01.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.222+0000 7fb67e000640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880719c0 0x7fb6881f5100 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb6881579d0 tx=0x7fb67400f9d0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.222+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb67801a070 con 0x7fb6880c7750 2024-04-24T22:41:01.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.223+0000 7fb67e000640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb66806b910 0x7fb66806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.223+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb67808a2a0 con 0x7fb6880c7750 2024-04-24T22:41:01.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.223+0000 7fb685400640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb66806b910 0x7fb66806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.224+0000 7fb685400640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb66806b910 0x7fb66806ddd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb6881f56b0 tx=0x7fb674007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:01.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.236+0000 7fb68d000640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6881f4580 0x7fb68814b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.236+0000 7fb68d000640 1 -- 172.21.15.47:0/3485981709 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb6880c1ac0 con 0x7fb6881f4580 2024-04-24T22:41:01.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.236+0000 7fb686800640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6881f4580 0x7fb68814b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.237+0000 7fb686800640 1 -- 172.21.15.47:0/3485981709 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb6881f4de0 con 0x7fb6881f4580 2024-04-24T22:41:01.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.237+0000 7fb686800640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6881f4580 0x7fb68814b3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb680009ee0 tx=0x7fb680004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:01.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.237+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680009450 con 0x7fb6881f4580 2024-04-24T22:41:01.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.237+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5208 (secure 0 0 0) 0x7fb68003c830 con 0x7fb6881f4580 2024-04-24T22:41:01.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.237+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6881f4580 msgr2=0x7fb68814b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.237+0000 7fb67e000640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6881f4580 0x7fb68814b3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb680009ee0 tx=0x7fb680004a40 comp rx=0 tx=0).stop 2024-04-24T22:41:01.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.237+0000 7fb67e000640 1 -- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6881f4580 msgr2=0x7fb68814b3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:01.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.237+0000 7fb67e000640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6881f4580 0x7fb68814b3c0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb680009ee0 tx=0x7fb680004a40 comp rx=0 tx=0).stop 2024-04-24T22:41:01.240 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:01.240 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 10, 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 102805, 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:01.241 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:01.242 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:01.244 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:01.245 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:01.246 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.247 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:01.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:01.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:01.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:01.251 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.252 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:01.253 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:01.254 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:01.254 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:01.254 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:01.254 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:01.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:01.254 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:01.254 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:01.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.238+0000 7fb68d000640 1 -- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb66806b910 msgr2=0x7fb66806ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.238+0000 7fb68d000640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb66806b910 0x7fb66806ddd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb6881f56b0 tx=0x7fb674007000 comp rx=0 tx=0).stop 2024-04-24T22:41:01.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.238+0000 7fb68d000640 1 -- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c7750 msgr2=0x7fb688153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.238+0000 7fb68d000640 1 --2- 172.21.15.47:0/3485981709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880c7750 0x7fb688153db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c3a0 tx=0x7fb67800c860 comp rx=0 tx=0).stop 2024-04-24T22:41:01.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.238+0000 7fb686800640 1 -- 172.21.15.47:0/3485981709 reap_dead start 2024-04-24T22:41:01.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.238+0000 7fb68d000640 1 -- 172.21.15.47:0/3485981709 shutdown_connections 2024-04-24T22:41:01.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.238+0000 7fb68d000640 1 -- 172.21.15.47:0/3485981709 >> 172.21.15.47:0/3485981709 conn(0x7fb68806e580 msgr2=0x7fb68814e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:01.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.238+0000 7fb68d000640 1 -- 172.21.15.47:0/3485981709 shutdown_connections 2024-04-24T22:41:01.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.238+0000 7fb68d000640 1 -- 172.21.15.47:0/3485981709 wait complete. 2024-04-24T22:41:01.263 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:41:01.263 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-24T22:41:01.264 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-24T22:41:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.445+0000 7f2286800640 1 Processor -- start 2024-04-24T22:41:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.446+0000 7f2286800640 1 -- start start 2024-04-24T22:41:01.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.447+0000 7f2286800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2280072c60 0x7f22800c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.447+0000 7f2286800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2280073040 0x7f22801411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.447+0000 7f2286800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f228015c3d0 con 0x7f228006bb90 2024-04-24T22:41:01.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.447+0000 7f2286800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2280158bf0 con 0x7f2280073040 2024-04-24T22:41:01.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.447+0000 7f2286800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f228015c6b0 con 0x7f2280072c60 2024-04-24T22:41:01.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.448+0000 7f2284a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2280073040 0x7f22801411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.448+0000 7f2285400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2280072c60 0x7f22800c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.448+0000 7f2285400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2280072c60 0x7f22800c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:33464/0 (socket says 172.21.15.47:33464) 2024-04-24T22:41:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.448+0000 7f2285400640 1 -- 172.21.15.47:0/3407916393 learned_addr learned my addr 172.21.15.47:0/3407916393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.448+0000 7f2285e00640 1 --1- 172.21.15.47:0/3407916393 >> v1:172.21.15.47:6789/0 conn(0x7f228006bb90 0x7f228015b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48196/0 (socket says 172.21.15.47:48196) 2024-04-24T22:41:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.448+0000 7f2285400640 1 -- 172.21.15.47:0/3407916393 >> v2:172.21.15.47:3300/0 conn(0x7f2280073040 msgr2=0x7f22801411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.448+0000 7f2285400640 1 --2- 172.21.15.47:0/3407916393 >> v2:172.21.15.47:3300/0 conn(0x7f2280073040 0x7f22801411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.448+0000 7f2285400640 1 -- 172.21.15.47:0/3407916393 >> v1:172.21.15.47:6789/0 conn(0x7f228006bb90 legacy=0x7f228015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.448+0000 7f2285400640 1 -- 172.21.15.47:0/3407916393 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22801585a0 con 0x7f2280072c60 2024-04-24T22:41:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.450+0000 7f2285400640 1 --2- 172.21.15.47:0/3407916393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2280072c60 0x7f22800c2d60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f228006b280 tx=0x7f227402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b08e640be66e3b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.450+0000 7f227fe00640 1 -- 172.21.15.47:0/3407916393 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274037070 con 0x7f2280072c60 2024-04-24T22:41:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.450+0000 7f227fe00640 1 -- 172.21.15.47:0/3407916393 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2274032440 con 0x7f2280072c60 2024-04-24T22:41:01.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.450+0000 7f227fe00640 1 -- 172.21.15.47:0/3407916393 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274032810 con 0x7f2280072c60 2024-04-24T22:41:01.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.451+0000 7f2286800640 1 -- 172.21.15.47:0/3407916393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2280072c60 msgr2=0x7f22800c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.451+0000 7f2286800640 1 --2- 172.21.15.47:0/3407916393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2280072c60 0x7f22800c2d60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f228006b280 tx=0x7f227402a270 comp rx=0 tx=0).stop 2024-04-24T22:41:01.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.451+0000 7f2286800640 1 -- 172.21.15.47:0/3407916393 shutdown_connections 2024-04-24T22:41:01.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.451+0000 7f2286800640 1 --2- 172.21.15.47:0/3407916393 >> v2:172.21.15.47:3300/0 conn(0x7f2280073040 0x7f22801411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.451+0000 7f2286800640 1 --2- 172.21.15.47:0/3407916393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2280072c60 0x7f22800c2d60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.451+0000 7f2286800640 1 -- 172.21.15.47:0/3407916393 >> 172.21.15.47:0/3407916393 conn(0x7f22800cb900 msgr2=0x7f22800cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:01.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.451+0000 7f2286800640 1 -- 172.21.15.47:0/3407916393 shutdown_connections 2024-04-24T22:41:01.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.451+0000 7f2286800640 1 -- 172.21.15.47:0/3407916393 wait complete. 2024-04-24T22:41:01.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.453+0000 7f2286800640 1 Processor -- start 2024-04-24T22:41:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.453+0000 7f2286800640 1 -- start start 2024-04-24T22:41:01.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.454+0000 7f2286800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228006bb90 0x7f228015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.454+0000 7f2286800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2280072c60 0x7f228015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.454+0000 7f2286800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2280073040 0x7f22801542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.454+0000 7f2286800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f22800c32a0 con 0x7f228006bb90 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.454+0000 7f2286800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2280158270 con 0x7f2280073040 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.454+0000 7f2286800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2280159080 con 0x7f2280072c60 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.454+0000 7f2285400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228006bb90 0x7f228015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.454+0000 7f2285400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228006bb90 0x7f228015a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36568/0 (socket says 172.21.15.47:36568) 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.454+0000 7f2284a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2280072c60 0x7f228015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.454+0000 7f2285400640 1 -- 172.21.15.47:0/3463608829 learned_addr learned my addr 172.21.15.47:0/3463608829 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.455+0000 7f2285e00640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2280073040 0x7f22801542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.455+0000 7f2285400640 1 -- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2280072c60 msgr2=0x7f228015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.455+0000 7f2285400640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2280072c60 0x7f228015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.455+0000 7f2285400640 1 -- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2280073040 msgr2=0x7f22801542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.455+0000 7f2285400640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2280073040 0x7f22801542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.455+0000 7f2285400640 1 -- 172.21.15.47:0/3463608829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22800725f0 con 0x7f228006bb90 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.455+0000 7f2284a00640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2280072c60 0x7f228015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.455+0000 7f2285400640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228006bb90 0x7f228015a6f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f227402a2b0 tx=0x7f22740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.456+0000 7f227d600640 1 -- 172.21.15.47:0/3463608829 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274037040 con 0x7f228006bb90 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.456+0000 7f227d600640 1 -- 172.21.15.47:0/3463608829 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2274042070 con 0x7f228006bb90 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.456+0000 7f227d600640 1 -- 172.21.15.47:0/3463608829 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274030880 con 0x7f228006bb90 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.456+0000 7f2286800640 1 -- 172.21.15.47:0/3463608829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2280153400 con 0x7f228006bb90 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.456+0000 7f2286800640 1 -- 172.21.15.47:0/3463608829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22801539c0 con 0x7f228006bb90 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.457+0000 7f2286800640 1 -- 172.21.15.47:0/3463608829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2280068d50 con 0x7f228006bb90 2024-04-24T22:41:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.458+0000 7f227d600640 1 -- 172.21.15.47:0/3463608829 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2274030a20 con 0x7f228006bb90 2024-04-24T22:41:01.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.459+0000 7f227d600640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f226006be30 0x7f226006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:01.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.459+0000 7f227d600640 1 -- 172.21.15.47:0/3463608829 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f22740ab770 con 0x7f228006bb90 2024-04-24T22:41:01.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.460+0000 7f2284a00640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f226006be30 0x7f226006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:01.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.461+0000 7f2284a00640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f226006be30 0x7f226006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f22800c23c0 tx=0x7f226c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:01.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.466+0000 7f227d600640 1 -- 172.21.15.47:0/3463608829 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2274079920 con 0x7f228006bb90 2024-04-24T22:41:01.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.766+0000 7f2286800640 1 -- 172.21.15.47:0/3463608829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f228004f5e0 con 0x7f228006bb90 2024-04-24T22:41:01.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.767+0000 7f227d600640 1 -- 172.21.15.47:0/3463608829 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f227407dc20 con 0x7f228006bb90 2024-04-24T22:41:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.770+0000 7f2286800640 1 -- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f226006be30 msgr2=0x7f226006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.770+0000 7f2286800640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f226006be30 0x7f226006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f22800c23c0 tx=0x7f226c009190 comp rx=0 tx=0).stop 2024-04-24T22:41:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.771+0000 7f2286800640 1 -- 172.21.15.47:0/3463608829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228006bb90 msgr2=0x7f228015a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.771+0000 7f2286800640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228006bb90 0x7f228015a6f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f227402a2b0 tx=0x7f22740046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.771+0000 7f2286800640 1 -- 172.21.15.47:0/3463608829 shutdown_connections 2024-04-24T22:41:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.771+0000 7f2286800640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f226006be30 0x7f226006e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.771+0000 7f2286800640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2280073040 0x7f22801542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.771+0000 7f2286800640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2280072c60 0x7f228015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.771+0000 7f2286800640 1 --2- 172.21.15.47:0/3463608829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f228006bb90 0x7f228015a6f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.771+0000 7f2286800640 1 -- 172.21.15.47:0/3463608829 >> 172.21.15.47:0/3463608829 conn(0x7f22800cb900 msgr2=0x7f22800c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.771+0000 7f2286800640 1 -- 172.21.15.47:0/3463608829 shutdown_connections 2024-04-24T22:41:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:01.771+0000 7f2286800640 1 -- 172.21.15.47:0/3463608829 wait complete. 2024-04-24T22:41:01.791 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-24T22:41:10.193 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:41:10.202 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:41:11.792 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-24T22:41:11.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:41:11.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.977+0000 7fa319e00640 1 Processor -- start 2024-04-24T22:41:11.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.977+0000 7fa319e00640 1 -- start start 2024-04-24T22:41:11.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.979+0000 7fa319e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3140c67e0 0x7fa3140cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.979+0000 7fa319e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3140c6bc0 0x7fa31415a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.979+0000 7fa319e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c7d20 0x7fa314066ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.979+0000 7fa319e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa3140cb690 con 0x7fa3140c6bc0 2024-04-24T22:41:11.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.979+0000 7fa319e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa31415aeb0 con 0x7fa3140c67e0 2024-04-24T22:41:11.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.979+0000 7fa319e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa31415b190 con 0x7fa3140c7d20 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa313e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3140c6bc0 0x7fa31415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa319400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c7d20 0x7fa314066ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa318a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3140c67e0 0x7fa3140cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa319400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c7d20 0x7fa314066ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:42186/0 (socket says 172.21.15.47:42186) 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa319400640 1 -- 172.21.15.47:0/3895455341 learned_addr learned my addr 172.21.15.47:0/3895455341 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa313e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3140c6bc0 0x7fa31415a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34556/0 (socket says 172.21.15.47:34556) 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa313e00640 1 -- 172.21.15.47:0/3895455341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c7d20 msgr2=0x7fa314066ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa313e00640 1 --2- 172.21.15.47:0/3895455341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c7d20 0x7fa314066ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa313e00640 1 -- 172.21.15.47:0/3895455341 >> v2:172.21.15.113:3300/0 conn(0x7fa3140c67e0 msgr2=0x7fa3140cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa313e00640 1 --2- 172.21.15.47:0/3895455341 >> v2:172.21.15.113:3300/0 conn(0x7fa3140c67e0 0x7fa3140cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.980+0000 7fa313e00640 1 -- 172.21.15.47:0/3895455341 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3140cb930 con 0x7fa3140c6bc0 2024-04-24T22:41:11.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.981+0000 7fa313e00640 1 --2- 172.21.15.47:0/3895455341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c6bc0 0x7fa31415a590 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa314067c50 tx=0x7fa30002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b831433c763f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:11.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa313400640 1 -- 172.21.15.47:0/3895455341 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300037070 con 0x7fa3140c6bc0 2024-04-24T22:41:11.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa313400640 1 -- 172.21.15.47:0/3895455341 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa300032440 con 0x7fa3140c6bc0 2024-04-24T22:41:11.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa313400640 1 -- 172.21.15.47:0/3895455341 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300032810 con 0x7fa3140c6bc0 2024-04-24T22:41:11.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa319e00640 1 -- 172.21.15.47:0/3895455341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c6bc0 msgr2=0x7fa31415a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa319e00640 1 --2- 172.21.15.47:0/3895455341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c6bc0 0x7fa31415a590 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa314067c50 tx=0x7fa30002a1a0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa319e00640 1 -- 172.21.15.47:0/3895455341 shutdown_connections 2024-04-24T22:41:11.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa319e00640 1 --2- 172.21.15.47:0/3895455341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c7d20 0x7fa314066ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa319e00640 1 --2- 172.21.15.47:0/3895455341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c6bc0 0x7fa31415a590 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa319e00640 1 --2- 172.21.15.47:0/3895455341 >> v2:172.21.15.113:3300/0 conn(0x7fa3140c67e0 0x7fa3140cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa319e00640 1 -- 172.21.15.47:0/3895455341 >> 172.21.15.47:0/3895455341 conn(0x7fa3140c9410 msgr2=0x7fa3140c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:11.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa319e00640 1 -- 172.21.15.47:0/3895455341 shutdown_connections 2024-04-24T22:41:11.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.982+0000 7fa319e00640 1 -- 172.21.15.47:0/3895455341 wait complete. 2024-04-24T22:41:11.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.984+0000 7fa319e00640 1 Processor -- start 2024-04-24T22:41:11.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.984+0000 7fa319e00640 1 -- start start 2024-04-24T22:41:11.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa319e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c67e0 0x7fa3141efce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa319e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c6bc0 0x7fa314071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa319e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3140c7d20 0x7fa3141f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa319e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa3140cc670 con 0x7fa3140c67e0 2024-04-24T22:41:11.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa319e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa31415b470 con 0x7fa3140c7d20 2024-04-24T22:41:11.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa319e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa3140cc7f0 con 0x7fa3140c6bc0 2024-04-24T22:41:11.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa318a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c67e0 0x7fa3141efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa318a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c67e0 0x7fa3141efce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34570/0 (socket says 172.21.15.47:34570) 2024-04-24T22:41:11.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa318a00640 1 -- 172.21.15.47:0/3792138433 learned_addr learned my addr 172.21.15.47:0/3792138433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:11.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa313e00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c6bc0 0x7fa314071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa319400640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3140c7d20 0x7fa3141f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa318a00640 1 -- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c6bc0 msgr2=0x7fa314071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa318a00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c6bc0 0x7fa314071b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa318a00640 1 -- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3140c7d20 msgr2=0x7fa3141f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:11.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa318a00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3140c7d20 0x7fa3141f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:11.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa318a00640 1 -- 172.21.15.47:0/3792138433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3140726f0 con 0x7fa3140c67e0 2024-04-24T22:41:11.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.986+0000 7fa313e00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c6bc0 0x7fa314071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:11.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.987+0000 7fa318a00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c67e0 0x7fa3141efce0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e8c0 tx=0x7fa30800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.987+0000 7fa310c00640 1 -- 172.21.15.47:0/3792138433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308017070 con 0x7fa3140c67e0 2024-04-24T22:41:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.987+0000 7fa310c00640 1 -- 172.21.15.47:0/3792138433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30800c940 con 0x7fa3140c67e0 2024-04-24T22:41:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.987+0000 7fa310c00640 1 -- 172.21.15.47:0/3792138433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30800cd10 con 0x7fa3140c67e0 2024-04-24T22:41:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.987+0000 7fa319e00640 1 -- 172.21.15.47:0/3792138433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3141f1500 con 0x7fa3140c67e0 2024-04-24T22:41:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.987+0000 7fa319e00640 1 -- 172.21.15.47:0/3792138433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3141f1ac0 con 0x7fa3140c67e0 2024-04-24T22:41:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.988+0000 7fa319e00640 1 -- 172.21.15.47:0/3792138433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa31404f5e0 con 0x7fa3140c67e0 2024-04-24T22:41:11.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.989+0000 7fa310c00640 1 -- 172.21.15.47:0/3792138433 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa30801a070 con 0x7fa3140c67e0 2024-04-24T22:41:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.991+0000 7fa310c00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa2f406be30 0x7fa2f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.991+0000 7fa310c00640 1 -- 172.21.15.47:0/3792138433 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa30808af10 con 0x7fa3140c67e0 2024-04-24T22:41:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.991+0000 7fa313e00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa2f406be30 0x7fa2f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.992+0000 7fa313e00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa2f406be30 0x7fa2f406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa300034000 tx=0x7fa30002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:11.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:11.997+0000 7fa310c00640 1 -- 172.21.15.47:0/3792138433 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3080590c0 con 0x7fa3140c67e0 2024-04-24T22:41:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.288+0000 7fa319e00640 1 -- 172.21.15.47:0/3792138433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa3141f2450 con 0x7fa3140c67e0 2024-04-24T22:41:12.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.289+0000 7fa310c00640 1 -- 172.21.15.47:0/3792138433 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fa30805d3c0 con 0x7fa3140c67e0 2024-04-24T22:41:12.291 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:41:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 -- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa2f406be30 msgr2=0x7fa2f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa2f406be30 0x7fa2f406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa300034000 tx=0x7fa30002e000 comp rx=0 tx=0).stop 2024-04-24T22:41:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 -- 172.21.15.47:0/3792138433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c67e0 msgr2=0x7fa3141efce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c67e0 0x7fa3141efce0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e8c0 tx=0x7fa30800ed80 comp rx=0 tx=0).stop 2024-04-24T22:41:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 -- 172.21.15.47:0/3792138433 shutdown_connections 2024-04-24T22:41:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa2f406be30 0x7fa2f406e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3140c7d20 0x7fa3141f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3140c6bc0 0x7fa314071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 --2- 172.21.15.47:0/3792138433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3140c67e0 0x7fa3141efce0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 -- 172.21.15.47:0/3792138433 >> 172.21.15.47:0/3792138433 conn(0x7fa3140c9410 msgr2=0x7fa3140c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 -- 172.21.15.47:0/3792138433 shutdown_connections 2024-04-24T22:41:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.293+0000 7fa319e00640 1 -- 172.21.15.47:0/3792138433 wait complete. 2024-04-24T22:41:12.318 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-24T22:41:12.318 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-24T22:41:12.318 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-24T22:41:12.318 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:41:12.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.549+0000 7fc32e200640 1 Processor -- start 2024-04-24T22:41:12.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.549+0000 7fc32e200640 1 -- start start 2024-04-24T22:41:12.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.551+0000 7fc32e200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc328154560 0x7fc3280cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.551+0000 7fc32e200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc3280cb970 0x7fc3280ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.551+0000 7fc32e200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3280cbdc0 con 0x7fc328073280 2024-04-24T22:41:12.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.551+0000 7fc32e200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc32815b2c0 con 0x7fc3280cb970 2024-04-24T22:41:12.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.551+0000 7fc32e200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc32815da90 con 0x7fc328154560 2024-04-24T22:41:12.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.551+0000 7fc32ce00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc328154560 0x7fc3280cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.551+0000 7fc327e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc328073280 0x7fc3280cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34130/0 (socket says 172.21.15.47:34130) 2024-04-24T22:41:12.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.551+0000 7fc327e00640 1 -- 172.21.15.47:0/2286584249 learned_addr learned my addr 172.21.15.47:0/2286584249 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:12.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.551+0000 7fc32d800640 1 --2- 172.21.15.47:0/2286584249 >> v2:172.21.15.47:3300/0 conn(0x7fc3280cb970 0x7fc3280ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.552+0000 7fc32ce00640 1 -- 172.21.15.47:0/2286584249 >> v2:172.21.15.47:3300/0 conn(0x7fc3280cb970 msgr2=0x7fc3280ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.552+0000 7fc32ce00640 1 --2- 172.21.15.47:0/2286584249 >> v2:172.21.15.47:3300/0 conn(0x7fc3280cb970 0x7fc3280ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.552+0000 7fc32ce00640 1 -- 172.21.15.47:0/2286584249 >> v1:172.21.15.47:6789/0 conn(0x7fc328073280 legacy=0x7fc3280cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.552+0000 7fc32ce00640 1 -- 172.21.15.47:0/2286584249 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32815dd70 con 0x7fc328154560 2024-04-24T22:41:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.552+0000 7fc32d800640 1 --2- 172.21.15.47:0/2286584249 >> v2:172.21.15.47:3300/0 conn(0x7fc3280cb970 0x7fc3280ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.553+0000 7fc32ce00640 1 --2- 172.21.15.47:0/2286584249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc328154560 0x7fc3280cb430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cd790 tx=0x7fc31c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=539471509ee64f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.554+0000 7fc327400640 1 -- 172.21.15.47:0/2286584249 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c037070 con 0x7fc328154560 2024-04-24T22:41:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.554+0000 7fc327400640 1 -- 172.21.15.47:0/2286584249 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c032440 con 0x7fc328154560 2024-04-24T22:41:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.554+0000 7fc327400640 1 -- 172.21.15.47:0/2286584249 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c032810 con 0x7fc328154560 2024-04-24T22:41:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.555+0000 7fc32e200640 1 -- 172.21.15.47:0/2286584249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc328154560 msgr2=0x7fc3280cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.555+0000 7fc32e200640 1 --2- 172.21.15.47:0/2286584249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc328154560 0x7fc3280cb430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cd790 tx=0x7fc31c02a1a0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.555+0000 7fc32e200640 1 -- 172.21.15.47:0/2286584249 shutdown_connections 2024-04-24T22:41:12.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.555+0000 7fc32e200640 1 --2- 172.21.15.47:0/2286584249 >> v2:172.21.15.47:3300/0 conn(0x7fc3280cb970 0x7fc3280ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.555+0000 7fc32e200640 1 --2- 172.21.15.47:0/2286584249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc328154560 0x7fc3280cb430 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.555+0000 7fc32e200640 1 -- 172.21.15.47:0/2286584249 >> 172.21.15.47:0/2286584249 conn(0x7fc3280672c0 msgr2=0x7fc328072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:12.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.555+0000 7fc32e200640 1 -- 172.21.15.47:0/2286584249 shutdown_connections 2024-04-24T22:41:12.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.555+0000 7fc32e200640 1 -- 172.21.15.47:0/2286584249 wait complete. 2024-04-24T22:41:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.557+0000 7fc32e200640 1 Processor -- start 2024-04-24T22:41:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.557+0000 7fc32e200640 1 -- start start 2024-04-24T22:41:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc32e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc328073280 0x7fc3281e4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc32e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3280cb970 0x7fc328156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc32e200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc328154560 0x7fc3281f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc32e200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3280caad0 con 0x7fc3280cb970 2024-04-24T22:41:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc32e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc32815e330 con 0x7fc328073280 2024-04-24T22:41:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc32e200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc3280cd2b0 con 0x7fc328154560 2024-04-24T22:41:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc327e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3280cb970 0x7fc328156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc32d800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc328154560 0x7fc3281f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc32d800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc328154560 0x7fc3281f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:42210/0 (socket says 172.21.15.47:42210) 2024-04-24T22:41:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc32d800640 1 -- 172.21.15.47:0/3131892519 learned_addr learned my addr 172.21.15.47:0/3131892519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc327e00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3280cb970 0x7fc328156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34596/0 (socket says 172.21.15.47:34596) 2024-04-24T22:41:12.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.558+0000 7fc32ce00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc328073280 0x7fc3281e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc32ce00640 1 -- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc328154560 msgr2=0x7fc3281f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc32ce00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc328154560 0x7fc3281f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc32ce00640 1 -- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3280cb970 msgr2=0x7fc328156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc32ce00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3280cb970 0x7fc328156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc32ce00640 1 -- 172.21.15.47:0/3131892519 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3281580e0 con 0x7fc328073280 2024-04-24T22:41:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc32d800640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc328154560 0x7fc3281f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc327e00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3280cb970 0x7fc328156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc32ce00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc328073280 0x7fc3281e4410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc31c031000 tx=0x7fc31c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c032e60 con 0x7fc328073280 2024-04-24T22:41:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c042070 con 0x7fc328073280 2024-04-24T22:41:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.559+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c037040 con 0x7fc328073280 2024-04-24T22:41:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.560+0000 7fc32e200640 1 -- 172.21.15.47:0/3131892519 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3281583c0 con 0x7fc328073280 2024-04-24T22:41:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.560+0000 7fc32e200640 1 -- 172.21.15.47:0/3131892519 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc328158980 con 0x7fc328073280 2024-04-24T22:41:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.561+0000 7fc32e200640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc328150740 0x7fc3281f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.561+0000 7fc32e200640 1 -- 172.21.15.47:0/3131892519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3281f4000 con 0x7fc328150740 2024-04-24T22:41:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.561+0000 7fc327e00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc328150740 0x7fc3281f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.562+0000 7fc327e00640 1 -- 172.21.15.47:0/3131892519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3280ccef0 con 0x7fc328150740 2024-04-24T22:41:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.563+0000 7fc327e00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc328150740 0x7fc3281f4900 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3280c9f30 tx=0x7fc31400fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.563+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314019070 con 0x7fc328150740 2024-04-24T22:41:12.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.563+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc31c030b70 con 0x7fc328073280 2024-04-24T22:41:12.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.565+0000 7fc324c00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc30806bd30 0x7fc30806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.565+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc31c0ab800 con 0x7fc328073280 2024-04-24T22:41:12.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.565+0000 7fc32d800640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc30806bd30 0x7fc30806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.566+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc314005ac0 con 0x7fc328150740 2024-04-24T22:41:12.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.566+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc328150740 msgr2=0x7fc3281f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.566+0000 7fc324c00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc328150740 0x7fc3281f4900 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3280c9f30 tx=0x7fc31400fd40 comp rx=0 tx=0).stop 2024-04-24T22:41:12.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.566+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc328150740 msgr2=0x7fc3281f4900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:12.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.566+0000 7fc324c00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc328150740 0x7fc3281f4900 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3280c9f30 tx=0x7fc31400fd40 comp rx=0 tx=0).stop 2024-04-24T22:41:12.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.566+0000 7fc32d800640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc30806bd30 0x7fc30806e1f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc320003ea0 tx=0x7fc3200097f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.580+0000 7fc32e200640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32815c610 0x7fc32815ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.580+0000 7fc32e200640 1 -- 172.21.15.47:0/3131892519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc328158b50 con 0x7fc32815c610 2024-04-24T22:41:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.580+0000 7fc327e00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32815c610 0x7fc32815ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.581+0000 7fc327e00640 1 -- 172.21.15.47:0/3131892519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc3280c6080 con 0x7fc32815c610 2024-04-24T22:41:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.581+0000 7fc327e00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32815c610 0x7fc32815ca20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc314013000 tx=0x7fc314000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.581+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314007730 con 0x7fc32815c610 2024-04-24T22:41:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4557 (secure 0 0 0) 0x7fc314007730 con 0x7fc32815c610 2024-04-24T22:41:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32815c610 msgr2=0x7fc32815ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc324c00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32815c610 0x7fc32815ca20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc314013000 tx=0x7fc314000ed0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc324c00640 1 -- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32815c610 msgr2=0x7fc32815ca20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc324c00640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32815c610 0x7fc32815ca20 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc314013000 tx=0x7fc314000ed0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.585 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:41:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 10, 2024-04-24T22:41:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 12618, 2024-04-24T22:41:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:41:12.585 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 12, 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:12.586 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:12.587 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:12.588 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:12.589 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.590 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.592 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:12.593 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.594 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.595 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.596 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:12.598 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc32e200640 1 -- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc30806bd30 msgr2=0x7fc30806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc32e200640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc30806bd30 0x7fc30806e1f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc320003ea0 tx=0x7fc3200097f0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc32e200640 1 -- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc328073280 msgr2=0x7fc3281e4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc32e200640 1 --2- 172.21.15.47:0/3131892519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc328073280 0x7fc3281e4410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc31c031000 tx=0x7fc31c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc32d800640 1 -- 172.21.15.47:0/3131892519 reap_dead start 2024-04-24T22:41:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc32d800640 1 -- 172.21.15.47:0/3131892519 reap_dead start 2024-04-24T22:41:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc32e200640 1 -- 172.21.15.47:0/3131892519 shutdown_connections 2024-04-24T22:41:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc32e200640 1 -- 172.21.15.47:0/3131892519 >> 172.21.15.47:0/3131892519 conn(0x7fc3280672c0 msgr2=0x7fc3281e3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.582+0000 7fc32e200640 1 -- 172.21.15.47:0/3131892519 shutdown_connections 2024-04-24T22:41:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.583+0000 7fc32e200640 1 -- 172.21.15.47:0/3131892519 wait complete. 2024-04-24T22:41:12.606 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:41:12.606 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-24T22:41:12.607 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:41:12.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.786+0000 7ff141e00640 1 Processor -- start 2024-04-24T22:41:12.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.786+0000 7ff141e00640 1 -- start start 2024-04-24T22:41:12.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.787+0000 7ff141e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff13c0c9e50 0x7ff13c15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.787+0000 7ff141e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff13c15bae0 0x7ff13c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.787+0000 7ff141e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff13c15b5c0 con 0x7ff13c06bf80 2024-04-24T22:41:12.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff141e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff13c15eb30 con 0x7ff13c15bae0 2024-04-24T22:41:12.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff141e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff13c0cc5d0 con 0x7ff13c0c9e50 2024-04-24T22:41:12.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff140a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff13c0c9e50 0x7ff13c15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff141400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff13c06bf80 0x7ff13c06b160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:37488/0 (socket says 172.21.15.47:37488) 2024-04-24T22:41:12.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff141400640 1 -- 172.21.15.47:0/3286831820 learned_addr learned my addr 172.21.15.47:0/3286831820 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:12.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff13be00640 1 --2- 172.21.15.47:0/3286831820 >> v2:172.21.15.47:3300/0 conn(0x7ff13c15bae0 0x7ff13c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff13b400640 1 -- 172.21.15.47:0/3286831820 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 359449103 0 0) 0x7ff13c15b5c0 con 0x7ff13c06bf80 2024-04-24T22:41:12.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff140a00640 1 -- 172.21.15.47:0/3286831820 >> v2:172.21.15.47:3300/0 conn(0x7ff13c15bae0 msgr2=0x7ff13c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:12.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff140a00640 1 --2- 172.21.15.47:0/3286831820 >> v2:172.21.15.47:3300/0 conn(0x7ff13c15bae0 0x7ff13c15b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff140a00640 1 -- 172.21.15.47:0/3286831820 >> v1:172.21.15.113:6789/0 conn(0x7ff13c06bf80 legacy=0x7ff13c06b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.788+0000 7ff140a00640 1 -- 172.21.15.47:0/3286831820 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c15e290 con 0x7ff13c0c9e50 2024-04-24T22:41:12.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.790+0000 7ff140a00640 1 --2- 172.21.15.47:0/3286831820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c0c9e50 0x7ff13c15dbe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff13c08d680 tx=0x7ff13002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7df1b4ad1791512 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.790+0000 7ff13b400640 1 -- 172.21.15.47:0/3286831820 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130037070 con 0x7ff13c0c9e50 2024-04-24T22:41:12.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.790+0000 7ff13b400640 1 -- 172.21.15.47:0/3286831820 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff130032440 con 0x7ff13c0c9e50 2024-04-24T22:41:12.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.790+0000 7ff13b400640 1 -- 172.21.15.47:0/3286831820 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130032810 con 0x7ff13c0c9e50 2024-04-24T22:41:12.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.791+0000 7ff141e00640 1 -- 172.21.15.47:0/3286831820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c0c9e50 msgr2=0x7ff13c15dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.791+0000 7ff141e00640 1 --2- 172.21.15.47:0/3286831820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c0c9e50 0x7ff13c15dbe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff13c08d680 tx=0x7ff13002a250 comp rx=0 tx=0).stop 2024-04-24T22:41:12.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.791+0000 7ff141e00640 1 -- 172.21.15.47:0/3286831820 shutdown_connections 2024-04-24T22:41:12.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.791+0000 7ff141e00640 1 --2- 172.21.15.47:0/3286831820 >> v2:172.21.15.47:3300/0 conn(0x7ff13c15bae0 0x7ff13c15b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.791+0000 7ff141e00640 1 --2- 172.21.15.47:0/3286831820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c0c9e50 0x7ff13c15dbe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.791+0000 7ff141e00640 1 -- 172.21.15.47:0/3286831820 >> 172.21.15.47:0/3286831820 conn(0x7ff13c0cfd00 msgr2=0x7ff13c0d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:12.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.791+0000 7ff141e00640 1 -- 172.21.15.47:0/3286831820 shutdown_connections 2024-04-24T22:41:12.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.791+0000 7ff141e00640 1 -- 172.21.15.47:0/3286831820 wait complete. 2024-04-24T22:41:12.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.793+0000 7ff141e00640 1 Processor -- start 2024-04-24T22:41:12.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.794+0000 7ff141e00640 1 -- start start 2024-04-24T22:41:12.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.795+0000 7ff141e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c06bf80 0x7ff13c1f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.795+0000 7ff141e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff13c0c9e50 0x7ff13c1e6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.795+0000 7ff141e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff13c15bae0 0x7ff13c1e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.795+0000 7ff141e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c15a880 con 0x7ff13c0c9e50 2024-04-24T22:41:12.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.795+0000 7ff141e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c15e850 con 0x7ff13c06bf80 2024-04-24T22:41:12.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.795+0000 7ff141e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff13c15ab60 con 0x7ff13c15bae0 2024-04-24T22:41:12.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff140a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c06bf80 0x7ff13c1f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff140a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c06bf80 0x7ff13c1f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:50350/0 (socket says 172.21.15.47:50350) 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff141400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff13c15bae0 0x7ff13c1e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff140a00640 1 -- 172.21.15.47:0/1805554459 learned_addr learned my addr 172.21.15.47:0/1805554459 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff13be00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff13c0c9e50 0x7ff13c1e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff140a00640 1 -- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff13c15bae0 msgr2=0x7ff13c1e3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff140a00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff13c15bae0 0x7ff13c1e3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff140a00640 1 -- 172.21.15.47:0/1805554459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff13c0c9e50 msgr2=0x7ff13c1e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff140a00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff13c0c9e50 0x7ff13c1e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff140a00640 1 -- 172.21.15.47:0/1805554459 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c15b540 con 0x7ff13c06bf80 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff13be00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff13c0c9e50 0x7ff13c1e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.796+0000 7ff140a00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c06bf80 0x7ff13c1f4150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff130000c00 tx=0x7ff1300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.797+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130041070 con 0x7ff13c06bf80 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.797+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff13003c890 con 0x7ff13c06bf80 2024-04-24T22:41:12.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.797+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130037040 con 0x7ff13c06bf80 2024-04-24T22:41:12.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.797+0000 7ff141e00640 1 -- 172.21.15.47:0/1805554459 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c1f4690 con 0x7ff13c06bf80 2024-04-24T22:41:12.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.797+0000 7ff141e00640 1 -- 172.21.15.47:0/1805554459 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c1e6f60 con 0x7ff13c06bf80 2024-04-24T22:41:12.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.799+0000 7ff141e00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c155cd0 0x7ff13c0cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.799+0000 7ff141e00640 1 -- 172.21.15.47:0/1805554459 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c15a590 con 0x7ff13c155cd0 2024-04-24T22:41:12.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.799+0000 7ff13be00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c155cd0 0x7ff13c0cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.799+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff130044070 con 0x7ff13c06bf80 2024-04-24T22:41:12.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.799+0000 7ff13be00640 1 -- 172.21.15.47:0/1805554459 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff13c04f5e0 con 0x7ff13c155cd0 2024-04-24T22:41:12.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.800+0000 7ff13be00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c155cd0 0x7ff13c0cad80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0b2770 tx=0x7ff128011a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.800+0000 7ff138c00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff11c06be30 0x7ff11c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.800+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff1300ab990 con 0x7ff13c06bf80 2024-04-24T22:41:12.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.800+0000 7ff141400640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff11c06be30 0x7ff11c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.801+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128015870 con 0x7ff13c155cd0 2024-04-24T22:41:12.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.801+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff128015a10 con 0x7ff13c155cd0 2024-04-24T22:41:12.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.801+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c155cd0 msgr2=0x7ff13c0cad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.801+0000 7ff138c00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c155cd0 0x7ff13c0cad80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0b2770 tx=0x7ff128011a60 comp rx=0 tx=0).stop 2024-04-24T22:41:12.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.801+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c155cd0 msgr2=0x7ff13c0cad80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:12.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.801+0000 7ff138c00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c155cd0 0x7ff13c0cad80 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0b2770 tx=0x7ff128011a60 comp rx=0 tx=0).stop 2024-04-24T22:41:12.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.801+0000 7ff141400640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff11c06be30 0x7ff11c06e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff1340018a0 tx=0x7ff134001830 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.814+0000 7ff141e00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c157680 0x7ff13c157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.814+0000 7ff141e00640 1 -- 172.21.15.47:0/1805554459 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c0cb760 con 0x7ff13c157680 2024-04-24T22:41:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.815+0000 7ff13be00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c157680 0x7ff13c157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.815+0000 7ff13be00640 1 -- 172.21.15.47:0/1805554459 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff13c1e7240 con 0x7ff13c157680 2024-04-24T22:41:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.815+0000 7ff13be00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c157680 0x7ff13c157a60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0b2770 tx=0x7ff1280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.816+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128004440 con 0x7ff13c157680 2024-04-24T22:41:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.816+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5066 (secure 0 0 0) 0x7ff128026070 con 0x7ff13c157680 2024-04-24T22:41:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.816+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c157680 msgr2=0x7ff13c157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.816+0000 7ff138c00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c157680 0x7ff13c157a60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0b2770 tx=0x7ff1280046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.816+0000 7ff138c00640 1 -- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c157680 msgr2=0x7ff13c157a60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.816+0000 7ff138c00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c157680 0x7ff13c157a60 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0b2770 tx=0x7ff1280046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.819 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:12.819 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:12.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:12.819 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:41:12.820 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 10, 2024-04-24T22:41:12.820 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 114383, 2024-04-24T22:41:12.820 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:12.820 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:12.820 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:41:12.820 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:41:12.820 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.820 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 12, 2024-04-24T22:41:12.820 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:12.821 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:12.822 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:12.822 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:12.822 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.822 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:12.822 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:12.822 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:12.822 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:12.823 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:12.823 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:12.823 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:12.823 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:12.823 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:12.824 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:12.825 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:12.825 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:12.825 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:12.825 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.825 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:12.825 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:12.825 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:12.825 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:12.825 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:12.826 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:12.829 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:12.830 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:12.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.832 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:12.833 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:12.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.834 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:12.835 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:41:12.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:12.836 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:12.836 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:12.836 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:12.836 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:12.836 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:12.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:12.836 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:12.836 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.817+0000 7ff141e00640 1 -- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff11c06be30 msgr2=0x7ff11c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.817+0000 7ff141e00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff11c06be30 0x7ff11c06e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff1340018a0 tx=0x7ff134001830 comp rx=0 tx=0).stop 2024-04-24T22:41:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.817+0000 7ff141e00640 1 -- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c06bf80 msgr2=0x7ff13c1f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.817+0000 7ff141e00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c06bf80 0x7ff13c1f4150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff130000c00 tx=0x7ff1300046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.817+0000 7ff141400640 1 -- 172.21.15.47:0/1805554459 reap_dead start 2024-04-24T22:41:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.817+0000 7ff141e00640 1 -- 172.21.15.47:0/1805554459 shutdown_connections 2024-04-24T22:41:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.817+0000 7ff141e00640 1 --2- 172.21.15.47:0/1805554459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff13c06bf80 0x7ff13c1f4150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.817+0000 7ff141e00640 1 -- 172.21.15.47:0/1805554459 >> 172.21.15.47:0/1805554459 conn(0x7ff13c0cfd00 msgr2=0x7ff13c0b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.817+0000 7ff141e00640 1 -- 172.21.15.47:0/1805554459 shutdown_connections 2024-04-24T22:41:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:12.817+0000 7ff141e00640 1 -- 172.21.15.47:0/1805554459 wait complete. 2024-04-24T22:41:12.839 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:41:12.839 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-24T22:41:12.839 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-24T22:41:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.022+0000 7f1ae8a00640 1 Processor -- start 2024-04-24T22:41:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.023+0000 7f1ae8a00640 1 -- start start 2024-04-24T22:41:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.025+0000 7f1ae8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae415dbe0 0x7f1ae406b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.025+0000 7f1ae8a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae415e290 con 0x7f1ae415bae0 2024-04-24T22:41:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.025+0000 7f1ae8a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae415e570 con 0x7f1ae40c9e50 2024-04-24T22:41:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.025+0000 7f1ae8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae406c690 con 0x7f1ae415dbe0 2024-04-24T22:41:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.025+0000 7f1ae2a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1ae415bae0 0x7f1ae406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34136/0 (socket says 172.21.15.47:34136) 2024-04-24T22:41:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.025+0000 7f1ae2a00640 1 -- 172.21.15.47:0/315050014 learned_addr learned my addr 172.21.15.47:0/315050014 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.025+0000 7f1ae3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1ae40c9e50 0x7f1ae415b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:37502/0 (socket says 172.21.15.47:37502) 2024-04-24T22:41:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.025+0000 7f1ae3e00640 1 --2- 172.21.15.47:0/315050014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae415dbe0 0x7f1ae406b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.026+0000 7f1ae2000640 1 -- 172.21.15.47:0/315050014 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1229009026 0 0) 0x7f1ae415e290 con 0x7f1ae415bae0 2024-04-24T22:41:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.026+0000 7f1ae3e00640 1 -- 172.21.15.47:0/315050014 >> v1:172.21.15.113:6789/0 conn(0x7f1ae40c9e50 legacy=0x7f1ae415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.026+0000 7f1ae3e00640 1 -- 172.21.15.47:0/315050014 >> v1:172.21.15.47:6789/0 conn(0x7f1ae415bae0 legacy=0x7f1ae406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.026+0000 7f1ae3e00640 1 -- 172.21.15.47:0/315050014 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae415e850 con 0x7f1ae415dbe0 2024-04-24T22:41:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.027+0000 7f1ae3e00640 1 --2- 172.21.15.47:0/315050014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae415dbe0 0x7f1ae406b160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009dc0 tx=0x7f1adc02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f441af94391ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.028+0000 7f1ae2000640 1 -- 172.21.15.47:0/315050014 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc007740 con 0x7f1ae415dbe0 2024-04-24T22:41:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.028+0000 7f1ae2000640 1 -- 172.21.15.47:0/315050014 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc0078e0 con 0x7f1ae415dbe0 2024-04-24T22:41:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.028+0000 7f1ae2000640 1 -- 172.21.15.47:0/315050014 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc007cb0 con 0x7f1ae415dbe0 2024-04-24T22:41:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.028+0000 7f1ae8a00640 1 -- 172.21.15.47:0/315050014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae415dbe0 msgr2=0x7f1ae406b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.028+0000 7f1ae8a00640 1 --2- 172.21.15.47:0/315050014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae415dbe0 0x7f1ae406b160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009dc0 tx=0x7f1adc02a680 comp rx=0 tx=0).stop 2024-04-24T22:41:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.028+0000 7f1ae8a00640 1 -- 172.21.15.47:0/315050014 shutdown_connections 2024-04-24T22:41:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.028+0000 7f1ae8a00640 1 --2- 172.21.15.47:0/315050014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae415dbe0 0x7f1ae406b160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.028+0000 7f1ae8a00640 1 -- 172.21.15.47:0/315050014 >> 172.21.15.47:0/315050014 conn(0x7f1ae40cfd00 msgr2=0x7f1ae40d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.028+0000 7f1ae8a00640 1 -- 172.21.15.47:0/315050014 shutdown_connections 2024-04-24T22:41:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.028+0000 7f1ae8a00640 1 -- 172.21.15.47:0/315050014 wait complete. 2024-04-24T22:41:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.031+0000 7f1ae8a00640 1 Processor -- start 2024-04-24T22:41:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.031+0000 7f1ae8a00640 1 -- start start 2024-04-24T22:41:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.032+0000 7f1ae8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40c9e50 0x7f1ae40c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae8a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae415bae0 0x7f1ae414fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ae415dbe0 0x7f1ae41f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae8a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae40cc5d0 con 0x7f1ae40c9e50 2024-04-24T22:41:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae415eb30 con 0x7f1ae415dbe0 2024-04-24T22:41:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae8a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae415ab60 con 0x7f1ae415bae0 2024-04-24T22:41:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40c9e50 0x7f1ae40c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40c9e50 0x7f1ae40c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34626/0 (socket says 172.21.15.47:34626) 2024-04-24T22:41:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae3400640 1 -- 172.21.15.47:0/15340603 learned_addr learned my addr 172.21.15.47:0/15340603 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae3e00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ae415dbe0 0x7f1ae41f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae2a00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae415bae0 0x7f1ae414fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae3400640 1 -- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae415bae0 msgr2=0x7f1ae414fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.033+0000 7f1ae3400640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae415bae0 0x7f1ae414fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.034+0000 7f1ae3400640 1 -- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ae415dbe0 msgr2=0x7f1ae41f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.034+0000 7f1ae3400640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ae415dbe0 0x7f1ae41f0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.034+0000 7f1ae3400640 1 -- 172.21.15.47:0/15340603 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae41f0370 con 0x7f1ae40c9e50 2024-04-24T22:41:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.034+0000 7f1ae3e00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ae415dbe0 0x7f1ae41f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.034+0000 7f1ae3400640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40c9e50 0x7f1ae40c8600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400c780 tx=0x7f1ad400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.034+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4018070 con 0x7f1ae40c9e50 2024-04-24T22:41:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.034+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad40072d0 con 0x7f1ae40c9e50 2024-04-24T22:41:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.034+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad40047e0 con 0x7f1ae40c9e50 2024-04-24T22:41:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.035+0000 7f1ae8a00640 1 -- 172.21.15.47:0/15340603 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae414f140 con 0x7f1ae40c9e50 2024-04-24T22:41:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.035+0000 7f1ae8a00640 1 -- 172.21.15.47:0/15340603 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae414f700 con 0x7f1ae40c9e50 2024-04-24T22:41:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.036+0000 7f1ae8a00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae41f0540 0x7f1ae414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.036+0000 7f1ae8a00640 1 -- 172.21.15.47:0/15340603 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae40c9070 con 0x7f1ae41f0540 2024-04-24T22:41:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.037+0000 7f1ae2a00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae41f0540 0x7f1ae414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.037+0000 7f1ae2a00640 1 -- 172.21.15.47:0/15340603 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ae4154c80 con 0x7f1ae41f0540 2024-04-24T22:41:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.038+0000 7f1ae2a00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae41f0540 0x7f1ae414de80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ae41ede90 tx=0x7f1ad0001970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.038+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000f5e0 con 0x7f1ae41f0540 2024-04-24T22:41:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.038+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1ad0018070 con 0x7f1ae41f0540 2024-04-24T22:41:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.038+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae41f0540 msgr2=0x7f1ae414de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.038+0000 7f1adb400640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae41f0540 0x7f1ae414de80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ae41ede90 tx=0x7f1ad0001970 comp rx=0 tx=0).stop 2024-04-24T22:41:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.038+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae41f0540 msgr2=0x7f1ae414de80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.038+0000 7f1adb400640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae41f0540 0x7f1ae414de80 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ae41ede90 tx=0x7f1ad0001970 comp rx=0 tx=0).stop 2024-04-24T22:41:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.039+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ad4004980 con 0x7f1ae40c9e50 2024-04-24T22:41:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.040+0000 7f1adb400640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1ac406ba60 0x7f1ac406df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.040+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f1ad408a1f0 con 0x7f1ae40c9e50 2024-04-24T22:41:13.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.040+0000 7f1ae2a00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1ac406ba60 0x7f1ac406df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.041+0000 7f1ae2a00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1ac406ba60 0x7f1ac406df20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0014000 tx=0x7f1ad0009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.052+0000 7f1ae8a00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae40b2b70 0x7f1ae41f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.052+0000 7f1ae8a00640 1 -- 172.21.15.47:0/15340603 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae404f5e0 con 0x7f1ae40b2b70 2024-04-24T22:41:13.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.052+0000 7f1ae3e00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae40b2b70 0x7f1ae41f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.053+0000 7f1ae3e00640 1 -- 172.21.15.47:0/15340603 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1ae40cb760 con 0x7f1ae40b2b70 2024-04-24T22:41:13.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.053+0000 7f1ae3e00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae40b2b70 0x7f1ae41f37d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ae408d1c0 tx=0x7f1adc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.053+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc007e40 con 0x7f1ae40b2b70 2024-04-24T22:41:13.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.053+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5209 (secure 0 0 0) 0x7f1adc007e40 con 0x7f1ae40b2b70 2024-04-24T22:41:13.055 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:41:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 10, 2024-04-24T22:41:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 114620, 2024-04-24T22:41:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:41:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:41:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 12, 2024-04-24T22:41:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:41:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:41:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:13.071 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:13.072 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:13.072 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:13.072 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:13.072 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:13.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:13.072 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:13.072 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae40b2b70 msgr2=0x7f1ae41f37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1adb400640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae40b2b70 0x7f1ae41f37d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ae408d1c0 tx=0x7f1adc004a40 comp rx=0 tx=0).stop 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1adb400640 1 -- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae40b2b70 msgr2=0x7f1ae41f37d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1adb400640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ae40b2b70 0x7f1ae41f37d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1ae8a00640 1 -- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1ac406ba60 msgr2=0x7f1ac406df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1ae8a00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1ac406ba60 0x7f1ac406df20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0014000 tx=0x7f1ad0009000 comp rx=0 tx=0).stop 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1ae8a00640 1 -- 172.21.15.47:0/15340603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40c9e50 msgr2=0x7f1ae40c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1ae8a00640 1 --2- 172.21.15.47:0/15340603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ae40c9e50 0x7f1ae40c8600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400c780 tx=0x7f1ad400cc40 comp rx=0 tx=0).stop 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1ae3e00640 1 -- 172.21.15.47:0/15340603 reap_dead start 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1ae8a00640 1 -- 172.21.15.47:0/15340603 shutdown_connections 2024-04-24T22:41:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1ae8a00640 1 -- 172.21.15.47:0/15340603 >> 172.21.15.47:0/15340603 conn(0x7f1ae40cfd00 msgr2=0x7f1ae415b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.054+0000 7f1ae8a00640 1 -- 172.21.15.47:0/15340603 shutdown_connections 2024-04-24T22:41:13.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.055+0000 7f1ae8a00640 1 -- 172.21.15.47:0/15340603 wait complete. 2024-04-24T22:41:13.075 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:41:13.075 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-24T22:41:13.075 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-24T22:41:13.075 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-24T22:41:13.076 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-24T22:41:13.076 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-24T22:41:13.155 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:41:13.156 INFO:tasks.ceph.mon.c:Stopped 2024-04-24T22:41:13.157 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-24T22:41:13.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:41:13.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.350+0000 7f2a85800640 1 Processor -- start 2024-04-24T22:41:13.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.350+0000 7f2a85800640 1 -- start start 2024-04-24T22:41:13.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.352+0000 7f2a85800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 0x7f2a8015dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.352+0000 7f2a85800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2a8006bf80 0x7f2a8006b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.352+0000 7f2a85800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a8015e290 con 0x7f2a800c9e50 2024-04-24T22:41:13.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.352+0000 7f2a85800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2a8015eb30 con 0x7f2a8006bf80 2024-04-24T22:41:13.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.352+0000 7f2a85800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2a800cc5d0 con 0x7f2a8015bae0 2024-04-24T22:41:13.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.352+0000 7f2a77e00640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 msgr2=0x7f2a8015dbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:41:13.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.352+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 0x7f2a8015dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:13.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.352+0000 7f2a7fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2a800c9e50 0x7f2a8015b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:37518/0 (socket says 172.21.15.47:37518) 2024-04-24T22:41:13.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.352+0000 7f2a7fe00640 1 -- 172.21.15.47:0/1892879205 learned_addr learned my addr 172.21.15.47:0/1892879205 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:13.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.352+0000 7f2a84e00640 1 --2- 172.21.15.47:0/1892879205 >> v2:172.21.15.47:3300/0 conn(0x7f2a8006bf80 0x7f2a8006b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.353+0000 7f2a84e00640 1 -- 172.21.15.47:0/1892879205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 msgr2=0x7f2a8015dbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:41:13.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.353+0000 7f2a84e00640 1 --2- 172.21.15.47:0/1892879205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 0x7f2a8015dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.353+0000 7f2a84e00640 1 -- 172.21.15.47:0/1892879205 >> v1:172.21.15.113:6789/0 conn(0x7f2a800c9e50 legacy=0x7f2a8015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.353+0000 7f2a84e00640 1 -- 172.21.15.47:0/1892879205 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8015e570 con 0x7f2a8006bf80 2024-04-24T22:41:13.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.354+0000 7f2a84e00640 1 --2- 172.21.15.47:0/1892879205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a8006bf80 0x7f2a8006b160 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009dc0 tx=0x7f2a7802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d10fea2797082d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.354+0000 7f2a7f400640 1 -- 172.21.15.47:0/1892879205 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78007740 con 0x7f2a8006bf80 2024-04-24T22:41:13.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.354+0000 7f2a7f400640 1 -- 172.21.15.47:0/1892879205 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a780078e0 con 0x7f2a8006bf80 2024-04-24T22:41:13.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.354+0000 7f2a7f400640 1 -- 172.21.15.47:0/1892879205 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78007cb0 con 0x7f2a8006bf80 2024-04-24T22:41:13.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.355+0000 7f2a85800640 1 -- 172.21.15.47:0/1892879205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a8006bf80 msgr2=0x7f2a8006b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.355+0000 7f2a85800640 1 --2- 172.21.15.47:0/1892879205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a8006bf80 0x7f2a8006b160 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009dc0 tx=0x7f2a7802a730 comp rx=0 tx=0).stop 2024-04-24T22:41:13.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.355+0000 7f2a85800640 1 -- 172.21.15.47:0/1892879205 shutdown_connections 2024-04-24T22:41:13.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.355+0000 7f2a85800640 1 --2- 172.21.15.47:0/1892879205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a8006bf80 0x7f2a8006b160 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.355+0000 7f2a85800640 1 --2- 172.21.15.47:0/1892879205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 0x7f2a8015dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.355+0000 7f2a85800640 1 -- 172.21.15.47:0/1892879205 >> 172.21.15.47:0/1892879205 conn(0x7f2a800cfd00 msgr2=0x7f2a800d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.355+0000 7f2a85800640 1 -- 172.21.15.47:0/1892879205 shutdown_connections 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.355+0000 7f2a85800640 1 -- 172.21.15.47:0/1892879205 wait complete. 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.357+0000 7f2a85800640 1 Processor -- start 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.358+0000 7f2a85800640 1 -- start start 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.359+0000 7f2a85800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a8006bf80 0x7f2a80158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.359+0000 7f2a85800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a800c9e50 0x7f2a801f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.359+0000 7f2a85800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 0x7f2a800c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.359+0000 7f2a85800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8015ab60 con 0x7f2a800c9e50 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.359+0000 7f2a85800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8015a880 con 0x7f2a8006bf80 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.359+0000 7f2a85800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8006c430 con 0x7f2a8015bae0 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a800c9e50 0x7f2a801f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a8006bf80 0x7f2a80158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a84e00640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 msgr2=0x7f2a800c56e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:41:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a8006bf80 0x7f2a80158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:50442/0 (socket says 172.21.15.47:50442) 2024-04-24T22:41:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a7fe00640 1 -- 172.21.15.47:0/2023688783 learned_addr learned my addr 172.21.15.47:0/2023688783 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a800c9e50 0x7f2a801f0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34668/0 (socket says 172.21.15.47:34668) 2024-04-24T22:41:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a84e00640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 0x7f2a800c56e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a77e00640 1 -- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 msgr2=0x7f2a800c56e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a77e00640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 0x7f2a800c56e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a77e00640 1 -- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a8006bf80 msgr2=0x7f2a80158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a77e00640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a8006bf80 0x7f2a80158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.360+0000 7f2a77e00640 1 -- 172.21.15.47:0/2023688783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a80157210 con 0x7f2a800c9e50 2024-04-24T22:41:13.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.361+0000 7f2a77e00640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a800c9e50 0x7f2a801f0d20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800b450 tx=0x7f2a6800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.361+0000 7f2a7cc00640 1 -- 172.21.15.47:0/2023688783 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68003db0 con 0x7f2a800c9e50 2024-04-24T22:41:13.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.361+0000 7f2a7cc00640 1 -- 172.21.15.47:0/2023688783 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a68015070 con 0x7f2a800c9e50 2024-04-24T22:41:13.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.361+0000 7f2a7cc00640 1 -- 172.21.15.47:0/2023688783 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680052b0 con 0x7f2a800c9e50 2024-04-24T22:41:13.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.361+0000 7f2a85800640 1 -- 172.21.15.47:0/2023688783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a801f0840 con 0x7f2a800c9e50 2024-04-24T22:41:13.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.361+0000 7f2a85800640 1 -- 172.21.15.47:0/2023688783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a80157760 con 0x7f2a800c9e50 2024-04-24T22:41:13.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.362+0000 7f2a85800640 1 -- 172.21.15.47:0/2023688783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a801f0630 con 0x7f2a800c9e50 2024-04-24T22:41:13.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.364+0000 7f2a7cc00640 1 -- 172.21.15.47:0/2023688783 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a6800f070 con 0x7f2a800c9e50 2024-04-24T22:41:13.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.365+0000 7f2a7cc00640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2a5c06be10 0x7f2a5c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:13.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.365+0000 7f2a7cc00640 1 -- 172.21.15.47:0/2023688783 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f2a6808a6c0 con 0x7f2a800c9e50 2024-04-24T22:41:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.366+0000 7f2a7fe00640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2a5c06be10 0x7f2a5c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.367+0000 7f2a7fe00640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2a5c06be10 0x7f2a5c06e2d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a70005880 tx=0x7f2a70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.375+0000 7f2a7cc00640 1 -- 172.21.15.47:0/2023688783 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2a6800d040 con 0x7f2a800c9e50 2024-04-24T22:41:13.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.850+0000 7f2a85800640 1 -- 172.21.15.47:0/2023688783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2a800b4280 con 0x7f2a800c9e50 2024-04-24T22:41:13.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.851+0000 7f2a7cc00640 1 -- 172.21.15.47:0/2023688783 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f2a68058f50 con 0x7f2a800c9e50 2024-04-24T22:41:13.853 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:41:13.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.854+0000 7f2a85800640 1 -- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2a5c06be10 msgr2=0x7f2a5c06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.854+0000 7f2a85800640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2a5c06be10 0x7f2a5c06e2d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a70005880 tx=0x7f2a70009040 comp rx=0 tx=0).stop 2024-04-24T22:41:13.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.854+0000 7f2a85800640 1 -- 172.21.15.47:0/2023688783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a800c9e50 msgr2=0x7f2a801f0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:13.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.854+0000 7f2a85800640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a800c9e50 0x7f2a801f0d20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800b450 tx=0x7f2a6800b910 comp rx=0 tx=0).stop 2024-04-24T22:41:13.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.854+0000 7f2a85800640 1 -- 172.21.15.47:0/2023688783 shutdown_connections 2024-04-24T22:41:13.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.854+0000 7f2a85800640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2a5c06be10 0x7f2a5c06e2d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.854+0000 7f2a85800640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a8015bae0 0x7f2a800c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.854+0000 7f2a85800640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a800c9e50 0x7f2a801f0d20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.854+0000 7f2a85800640 1 --2- 172.21.15.47:0/2023688783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a8006bf80 0x7f2a80158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:13.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.854+0000 7f2a85800640 1 -- 172.21.15.47:0/2023688783 >> 172.21.15.47:0/2023688783 conn(0x7f2a800cfd00 msgr2=0x7f2a8015bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:13.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.855+0000 7f2a85800640 1 -- 172.21.15.47:0/2023688783 shutdown_connections 2024-04-24T22:41:13.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:13.855+0000 7f2a85800640 1 -- 172.21.15.47:0/2023688783 wait complete. 2024-04-24T22:41:16.875 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:41:17.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.090+0000 7ff192e00640 1 Processor -- start 2024-04-24T22:41:17.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.090+0000 7ff192e00640 1 -- start start 2024-04-24T22:41:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.091+0000 7ff192e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff18c154960 0x7ff18c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.091+0000 7ff192e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff18c15ab10 con 0x7ff18c15a680 2024-04-24T22:41:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.091+0000 7ff192e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff18c15adf0 con 0x7ff18c15e830 2024-04-24T22:41:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.091+0000 7ff192e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff18c15b0d0 con 0x7ff18c154960 2024-04-24T22:41:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.092+0000 7ff18b400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff18c15a680 0x7ff18c0c8590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46940/0 (socket says 172.21.15.47:46940) 2024-04-24T22:41:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.092+0000 7ff18b400640 1 -- 172.21.15.47:0/3064288585 learned_addr learned my addr 172.21.15.47:0/3064288585 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:17.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.092+0000 7ff18be00640 1 --2- 172.21.15.47:0/3064288585 >> v2:172.21.15.113:3300/0 conn(0x7ff18c154960 0x7ff18c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:17.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.092+0000 7ff18be00640 1 -- 172.21.15.47:0/3064288585 >> v1:172.21.15.113:6789/0 conn(0x7ff18c15e830 legacy=0x7ff18c0c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.092+0000 7ff18be00640 1 -- 172.21.15.47:0/3064288585 >> v1:172.21.15.47:6789/0 conn(0x7ff18c15a680 legacy=0x7ff18c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.092+0000 7ff18be00640 1 -- 172.21.15.47:0/3064288585 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18c15b3b0 con 0x7ff18c154960 2024-04-24T22:41:17.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.094+0000 7ff18be00640 1 --2- 172.21.15.47:0/3064288585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 0x7ff18c15e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0097a0 tx=0x7ff17c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7583c2d9e4cda1cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:17.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.094+0000 7ff18aa00640 1 -- 172.21.15.47:0/3064288585 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c037070 con 0x7ff18c154960 2024-04-24T22:41:17.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.094+0000 7ff18aa00640 1 -- 172.21.15.47:0/3064288585 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17c032440 con 0x7ff18c154960 2024-04-24T22:41:17.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.094+0000 7ff18aa00640 1 -- 172.21.15.47:0/3064288585 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c032810 con 0x7ff18c154960 2024-04-24T22:41:17.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.095+0000 7ff192e00640 1 -- 172.21.15.47:0/3064288585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 msgr2=0x7ff18c15e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.095+0000 7ff192e00640 1 --2- 172.21.15.47:0/3064288585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 0x7ff18c15e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0097a0 tx=0x7ff17c02a230 comp rx=0 tx=0).stop 2024-04-24T22:41:17.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.095+0000 7ff192e00640 1 -- 172.21.15.47:0/3064288585 shutdown_connections 2024-04-24T22:41:17.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.095+0000 7ff192e00640 1 --2- 172.21.15.47:0/3064288585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 0x7ff18c15e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.095+0000 7ff192e00640 1 -- 172.21.15.47:0/3064288585 >> 172.21.15.47:0/3064288585 conn(0x7ff18c06e580 msgr2=0x7ff18c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:17.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.095+0000 7ff192e00640 1 -- 172.21.15.47:0/3064288585 shutdown_connections 2024-04-24T22:41:17.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.096+0000 7ff192e00640 1 -- 172.21.15.47:0/3064288585 wait complete. 2024-04-24T22:41:17.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.098+0000 7ff192e00640 1 Processor -- start 2024-04-24T22:41:17.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.098+0000 7ff192e00640 1 -- start start 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.099+0000 7ff192e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 0x7ff18c155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.099+0000 7ff192e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c15e830 0x7ff18c1efed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.099+0000 7ff192e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff18c14b480 0x7ff18c14ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff192e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c0c8ca0 con 0x7ff18c15e830 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff192e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c154d40 con 0x7ff18c154960 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff192e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff18c0c8e20 con 0x7ff18c14b480 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff190e00640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff18c14b480 msgr2=0x7ff18c14ad10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff190e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff18c14b480 0x7ff18c14ad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff18be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 0x7ff18c155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff18be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 0x7ff18c155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:44732/0 (socket says 172.21.15.47:44732) 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff18be00640 1 -- 172.21.15.47:0/501239068 learned_addr learned my addr 172.21.15.47:0/501239068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff18b400640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c15e830 0x7ff18c1efed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:17.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff18be00640 1 -- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff18c14b480 msgr2=0x7ff18c14ad10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:17.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff18be00640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff18c14b480 0x7ff18c14ad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff18be00640 1 -- 172.21.15.47:0/501239068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c15e830 msgr2=0x7ff18c1efed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff18be00640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c15e830 0x7ff18c1efed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff18be00640 1 -- 172.21.15.47:0/501239068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18c14b250 con 0x7ff18c154960 2024-04-24T22:41:17.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.100+0000 7ff18b400640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c15e830 0x7ff18c1efed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.101+0000 7ff18be00640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 0x7ff18c155990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff17c035040 tx=0x7ff17c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.101+0000 7ff183e00640 1 -- 172.21.15.47:0/501239068 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c037040 con 0x7ff18c154960 2024-04-24T22:41:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.101+0000 7ff183e00640 1 -- 172.21.15.47:0/501239068 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17c030770 con 0x7ff18c154960 2024-04-24T22:41:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.101+0000 7ff183e00640 1 -- 172.21.15.47:0/501239068 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c030b80 con 0x7ff18c154960 2024-04-24T22:41:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.101+0000 7ff192e00640 1 -- 172.21.15.47:0/501239068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18c15ba50 con 0x7ff18c154960 2024-04-24T22:41:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.101+0000 7ff192e00640 1 -- 172.21.15.47:0/501239068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18c155190 con 0x7ff18c154960 2024-04-24T22:41:17.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.102+0000 7ff192e00640 1 -- 172.21.15.47:0/501239068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff18c15bf70 con 0x7ff18c154960 2024-04-24T22:41:17.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.104+0000 7ff183e00640 1 -- 172.21.15.47:0/501239068 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff17c044020 con 0x7ff18c154960 2024-04-24T22:41:17.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.105+0000 7ff183e00640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff16c06bde0 0x7ff16c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:17.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.105+0000 7ff183e00640 1 -- 172.21.15.47:0/501239068 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff17c0ab640 con 0x7ff18c154960 2024-04-24T22:41:17.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.105+0000 7ff18b400640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff16c06bde0 0x7ff16c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:17.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.106+0000 7ff18b400640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff16c06bde0 0x7ff16c06e2a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff18c151490 tx=0x7ff178009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:17.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.109+0000 7ff183e00640 1 -- 172.21.15.47:0/501239068 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff17c032e20 con 0x7ff18c154960 2024-04-24T22:41:17.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.474+0000 7ff192e00640 1 -- 172.21.15.47:0/501239068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff18c073b90 con 0x7ff18c154960 2024-04-24T22:41:17.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.475+0000 7ff183e00640 1 -- 172.21.15.47:0/501239068 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7ff17c079e40 con 0x7ff18c154960 2024-04-24T22:41:17.479 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:41:17.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.478+0000 7ff192e00640 1 -- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff16c06bde0 msgr2=0x7ff16c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.478+0000 7ff192e00640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff16c06bde0 0x7ff16c06e2a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff18c151490 tx=0x7ff178009040 comp rx=0 tx=0).stop 2024-04-24T22:41:17.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.478+0000 7ff192e00640 1 -- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 msgr2=0x7ff18c155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:17.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.478+0000 7ff192e00640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 0x7ff18c155990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff17c035040 tx=0x7ff17c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.478+0000 7ff192e00640 1 -- 172.21.15.47:0/501239068 shutdown_connections 2024-04-24T22:41:17.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.478+0000 7ff192e00640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff16c06bde0 0x7ff16c06e2a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.478+0000 7ff192e00640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff18c14b480 0x7ff18c14ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.478+0000 7ff192e00640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c15e830 0x7ff18c1efed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.478+0000 7ff192e00640 1 --2- 172.21.15.47:0/501239068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff18c154960 0x7ff18c155990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:17.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.478+0000 7ff192e00640 1 -- 172.21.15.47:0/501239068 >> 172.21.15.47:0/501239068 conn(0x7ff18c06e580 msgr2=0x7ff18c0c8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:17.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.479+0000 7ff192e00640 1 -- 172.21.15.47:0/501239068 shutdown_connections 2024-04-24T22:41:17.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:17.479+0000 7ff192e00640 1 -- 172.21.15.47:0/501239068 wait complete. 2024-04-24T22:41:20.501 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:41:20.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.666+0000 7f4073200640 1 Processor -- start 2024-04-24T22:41:20.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.667+0000 7f4073200640 1 -- start start 2024-04-24T22:41:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.668+0000 7f4073200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4074154560 0x7f40740cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.668+0000 7f4073200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40740cbdc0 con 0x7f4074073280 2024-04-24T22:41:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.668+0000 7f4073200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f407415b2c0 con 0x7f40740cb970 2024-04-24T22:41:20.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.668+0000 7f4073200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f407415da90 con 0x7f4074154560 2024-04-24T22:41:20.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.668+0000 7f4071e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4074154560 0x7f40740cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.668+0000 7f4071400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4074073280 0x7f40740cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46948/0 (socket says 172.21.15.47:46948) 2024-04-24T22:41:20.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.668+0000 7f4071400640 1 -- 172.21.15.47:0/3812610674 learned_addr learned my addr 172.21.15.47:0/3812610674 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:20.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.669+0000 7f4071e00640 1 -- 172.21.15.47:0/3812610674 >> v1:172.21.15.113:6789/0 conn(0x7f40740cb970 legacy=0x7f40740ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.669+0000 7f4071e00640 1 -- 172.21.15.47:0/3812610674 >> v1:172.21.15.47:6789/0 conn(0x7f4074073280 legacy=0x7f40740cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.669+0000 7f4071e00640 1 -- 172.21.15.47:0/3812610674 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407415dd70 con 0x7f4074154560 2024-04-24T22:41:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.670+0000 7f4071e00640 1 --2- 172.21.15.47:0/3812610674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074154560 0x7f40740cb430 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40640097a0 tx=0x7f406402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1816203dc8e56648 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.670+0000 7f4070a00640 1 -- 172.21.15.47:0/3812610674 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064037070 con 0x7f4074154560 2024-04-24T22:41:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.670+0000 7f4070a00640 1 -- 172.21.15.47:0/3812610674 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4064032440 con 0x7f4074154560 2024-04-24T22:41:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.670+0000 7f4070a00640 1 -- 172.21.15.47:0/3812610674 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064032810 con 0x7f4074154560 2024-04-24T22:41:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.671+0000 7f4073200640 1 -- 172.21.15.47:0/3812610674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074154560 msgr2=0x7f40740cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.671+0000 7f4073200640 1 --2- 172.21.15.47:0/3812610674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074154560 0x7f40740cb430 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40640097a0 tx=0x7f406402a230 comp rx=0 tx=0).stop 2024-04-24T22:41:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.671+0000 7f4073200640 1 -- 172.21.15.47:0/3812610674 shutdown_connections 2024-04-24T22:41:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.671+0000 7f4073200640 1 --2- 172.21.15.47:0/3812610674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074154560 0x7f40740cb430 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.671+0000 7f4073200640 1 -- 172.21.15.47:0/3812610674 >> 172.21.15.47:0/3812610674 conn(0x7f40740672c0 msgr2=0x7f4074072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.671+0000 7f4073200640 1 -- 172.21.15.47:0/3812610674 shutdown_connections 2024-04-24T22:41:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.671+0000 7f4073200640 1 -- 172.21.15.47:0/3812610674 wait complete. 2024-04-24T22:41:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.673+0000 7f4073200640 1 Processor -- start 2024-04-24T22:41:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.673+0000 7f4073200640 1 -- start start 2024-04-24T22:41:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4073200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074073280 0x7f40741595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4073200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40740cb970 0x7f407415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4073200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4074154560 0x7f4074158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4073200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f40740caad0 con 0x7f4074073280 2024-04-24T22:41:20.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4073200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f407415e330 con 0x7f4074154560 2024-04-24T22:41:20.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4073200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f407415ae80 con 0x7f40740cb970 2024-04-24T22:41:20.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4071e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074073280 0x7f40741595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4071400640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40740cb970 msgr2=0x7f407415c700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:41:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4071400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40740cb970 0x7f407415c700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4071e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074073280 0x7f40741595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37936/0 (socket says 172.21.15.47:37936) 2024-04-24T22:41:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4071e00640 1 -- 172.21.15.47:0/1550314964 learned_addr learned my addr 172.21.15.47:0/1550314964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.674+0000 7f4072800640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4074154560 0x7f4074158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.675+0000 7f4071e00640 1 -- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40740cb970 msgr2=0x7f407415c700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.675+0000 7f4071e00640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40740cb970 0x7f407415c700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.675+0000 7f4071e00640 1 -- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4074154560 msgr2=0x7f4074158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.675+0000 7f4071e00640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4074154560 0x7f4074158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.675+0000 7f4071e00640 1 -- 172.21.15.47:0/1550314964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407415d020 con 0x7f4074073280 2024-04-24T22:41:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.675+0000 7f4071e00640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074073280 0x7f40741595f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4064035040 tx=0x7f40640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.675+0000 7f406a000640 1 -- 172.21.15.47:0/1550314964 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064037040 con 0x7f4074073280 2024-04-24T22:41:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.675+0000 7f406a000640 1 -- 172.21.15.47:0/1550314964 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4064030770 con 0x7f4074073280 2024-04-24T22:41:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.675+0000 7f4073200640 1 -- 172.21.15.47:0/1550314964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40741f1160 con 0x7f4074073280 2024-04-24T22:41:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.675+0000 7f406a000640 1 -- 172.21.15.47:0/1550314964 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064030b80 con 0x7f4074073280 2024-04-24T22:41:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.676+0000 7f4073200640 1 -- 172.21.15.47:0/1550314964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40741f1720 con 0x7f4074073280 2024-04-24T22:41:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.677+0000 7f405f400640 1 -- 172.21.15.47:0/1550314964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4048005290 con 0x7f4074073280 2024-04-24T22:41:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.678+0000 7f406a000640 1 -- 172.21.15.47:0/1550314964 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4064044020 con 0x7f4074073280 2024-04-24T22:41:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.679+0000 7f406a000640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f405406bde0 0x7f405406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.679+0000 7f406a000640 1 -- 172.21.15.47:0/1550314964 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f40640ab5a0 con 0x7f4074073280 2024-04-24T22:41:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.679+0000 7f4071400640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f405406bde0 0x7f405406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.680+0000 7f4071400640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f405406bde0 0x7f405406e2a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4060007fd0 tx=0x7f4060007aa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.683+0000 7f406a000640 1 -- 172.21.15.47:0/1550314964 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4064032e20 con 0x7f4074073280 2024-04-24T22:41:20.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.960+0000 7f405f400640 1 -- 172.21.15.47:0/1550314964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4048005580 con 0x7f4074073280 2024-04-24T22:41:20.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.961+0000 7f406a000640 1 -- 172.21.15.47:0/1550314964 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f4064079da0 con 0x7f4074073280 2024-04-24T22:41:20.963 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 -- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f405406bde0 msgr2=0x7f405406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f405406bde0 0x7f405406e2a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4060007fd0 tx=0x7f4060007aa0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 -- 172.21.15.47:0/1550314964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074073280 msgr2=0x7f40741595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074073280 0x7f40741595f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4064035040 tx=0x7f40640046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 -- 172.21.15.47:0/1550314964 shutdown_connections 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f405406bde0 0x7f405406e2a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4074154560 0x7f4074158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40740cb970 0x7f407415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 --2- 172.21.15.47:0/1550314964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4074073280 0x7f40741595f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 -- 172.21.15.47:0/1550314964 >> 172.21.15.47:0/1550314964 conn(0x7f40740672c0 msgr2=0x7f407414eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 -- 172.21.15.47:0/1550314964 shutdown_connections 2024-04-24T22:41:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:20.963+0000 7f405f400640 1 -- 172.21.15.47:0/1550314964 wait complete. 2024-04-24T22:41:23.982 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:41:24.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.251+0000 7f8e93e00640 1 Processor -- start 2024-04-24T22:41:24.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.251+0000 7f8e93e00640 1 -- start start 2024-04-24T22:41:24.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.253+0000 7f8e93e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.253+0000 7f8e93e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e94071570 con 0x7f8e940c6a20 2024-04-24T22:41:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.253+0000 7f8e93e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e9415a8a0 con 0x7f8e9406e6f0 2024-04-24T22:41:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.253+0000 7f8e93e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8e9415b470 con 0x7f8e941514f0 2024-04-24T22:41:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.253+0000 7f8e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.254+0000 7f8e92000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8e9406e6f0 0x7f8e940c6310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:33152/0 (socket says 172.21.15.47:33152) 2024-04-24T22:41:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.254+0000 7f8e92000640 1 -- 172.21.15.47:0/1832053590 learned_addr learned my addr 172.21.15.47:0/1832053590 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.254+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 msgr2=0x7f8e9406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:41:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.254+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 msgr2=0x7f8e9406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.254+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.254+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.455+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.455+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 msgr2=0x7f8e9406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:41:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.455+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 msgr2=0x7f8e9406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.455+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.455+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:41:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.856+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.856+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 msgr2=0x7f8e9406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:41:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.856+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 msgr2=0x7f8e9406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.856+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:24.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:24.857+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:41:25.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:25.658+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:25.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:25.658+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 msgr2=0x7f8e9406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:41:25.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:25.658+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 msgr2=0x7f8e9406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:25.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:25.658+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:25.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:25.658+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:41:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.253+0000 7f8e8be00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 msgr2=0x7f8e9406e1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:41:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.253+0000 7f8e8be00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.113:3300/0 conn(0x7f8e941514f0 0x7f8e9406e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.253+0000 7f8e8be00640 1 -- 172.21.15.47:0/1832053590 >> v1:172.21.15.113:6789/0 conn(0x7f8e9406e6f0 legacy=0x7f8e940c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.253+0000 7f8e8be00640 1 -- 172.21.15.47:0/1832053590 >> v1:172.21.15.47:6789/0 conn(0x7f8e940c6a20 legacy=0x7f8e940707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.253+0000 7f8e8be00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.47:3300/0 conn(0x7f8e740036c0 0x7f8e74005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.253+0000 7f8e8be00640 1 --2- 172.21.15.47:0/1832053590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e74006180 0x7f8e74008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.253+0000 7f8e8be00640 1 -- 172.21.15.47:0/1832053590 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8e9415b470 con 0x7f8e740036c0 2024-04-24T22:41:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.253+0000 7f8e8be00640 1 -- 172.21.15.47:0/1832053590 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8e74008d30 con 0x7f8e74006180 2024-04-24T22:41:27.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.254+0000 7f8e92000640 1 -- 172.21.15.47:0/1832053590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e74006180 msgr2=0x7f8e74008640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:41:27.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.254+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.47:3300/0 conn(0x7f8e740036c0 0x7f8e74005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:27.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.254+0000 7f8e92000640 1 --2- 172.21.15.47:0/1832053590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e74006180 0x7f8e74008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:27.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.254+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.47:3300/0 conn(0x7f8e740036c0 msgr2=0x7f8e74005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:41:27.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.254+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> v2:172.21.15.47:3300/0 conn(0x7f8e740036c0 msgr2=0x7f8e74005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:27.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.254+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.47:3300/0 conn(0x7f8e740036c0 0x7f8e74005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.254+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.47:3300/0 conn(0x7f8e740036c0 0x7f8e74005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:27.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.455+0000 7f8e92000640 1 -- 172.21.15.47:0/1832053590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e74006180 msgr2=0x7f8e74008640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:41:27.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.455+0000 7f8e92000640 1 --2- 172.21.15.47:0/1832053590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e74006180 0x7f8e74008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:41:27.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.455+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> v2:172.21.15.47:3300/0 conn(0x7f8e740036c0 0x7f8e74005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:27.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.455+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e74006180 msgr2=0x7f8e74008640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:41:27.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.455+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e74006180 0x7f8e74008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.456+0000 7f8e92a00640 1 -- 172.21.15.47:0/1832053590 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e941518d0 con 0x7f8e740036c0 2024-04-24T22:41:27.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.457+0000 7f8e92a00640 1 --2- 172.21.15.47:0/1832053590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e740036c0 0x7f8e74005b40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8e84006fd0 tx=0x7f8e8402b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfde6eeb091208b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:27.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.457+0000 7f8e91600640 1 -- 172.21.15.47:0/1832053590 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84004030 con 0x7f8e740036c0 2024-04-24T22:41:27.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.457+0000 7f8e91600640 1 -- 172.21.15.47:0/1832053590 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e840041f0 con 0x7f8e740036c0 2024-04-24T22:41:27.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.458+0000 7f8e91600640 1 -- 172.21.15.47:0/1832053590 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e840043e0 con 0x7f8e740036c0 2024-04-24T22:41:27.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.458+0000 7f8e93e00640 1 -- 172.21.15.47:0/1832053590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e740036c0 msgr2=0x7f8e74005b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:27.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.458+0000 7f8e93e00640 1 --2- 172.21.15.47:0/1832053590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e740036c0 0x7f8e74005b40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8e84006fd0 tx=0x7f8e8402b470 comp rx=0 tx=0).stop 2024-04-24T22:41:27.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.458+0000 7f8e93400640 1 -- 172.21.15.47:0/1832053590 reap_dead start 2024-04-24T22:41:27.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.458+0000 7f8e93e00640 1 -- 172.21.15.47:0/1832053590 shutdown_connections 2024-04-24T22:41:27.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.458+0000 7f8e93e00640 1 --2- 172.21.15.47:0/1832053590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e740036c0 0x7f8e74005b40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.458+0000 7f8e93e00640 1 -- 172.21.15.47:0/1832053590 >> 172.21.15.47:0/1832053590 conn(0x7f8e940cb900 msgr2=0x7f8e940cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:27.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.458+0000 7f8e93e00640 1 -- 172.21.15.47:0/1832053590 shutdown_connections 2024-04-24T22:41:27.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.459+0000 7f8e93e00640 1 -- 172.21.15.47:0/1832053590 wait complete. 2024-04-24T22:41:27.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.460+0000 7f8e93e00640 1 Processor -- start 2024-04-24T22:41:27.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.461+0000 7f8e93e00640 1 -- start start 2024-04-24T22:41:27.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.461+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e740036c0 0x7f8e940ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.461+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e941594c0 0x7f8e940cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.461+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159ed0 0x7f8e94146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.461+0000 7f8e93e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9415b100 con 0x7f8e94159ed0 2024-04-24T22:41:27.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.461+0000 7f8e93e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9415ae20 con 0x7f8e740036c0 2024-04-24T22:41:27.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.461+0000 7f8e93e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8e940c7ba0 con 0x7f8e941594c0 2024-04-24T22:41:27.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.462+0000 7f8e92000640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e941594c0 msgr2=0x7f8e940cb170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:41:27.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.462+0000 7f8e92000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e941594c0 0x7f8e940cb170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:27.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.462+0000 7f8e93400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159ed0 0x7f8e94146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.462+0000 7f8e92a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e740036c0 0x7f8e940ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.462+0000 7f8e93400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159ed0 0x7f8e94146540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38544/0 (socket says 172.21.15.47:38544) 2024-04-24T22:41:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.462+0000 7f8e93400640 1 -- 172.21.15.47:0/2589814321 learned_addr learned my addr 172.21.15.47:0/2589814321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.462+0000 7f8e93400640 1 -- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e941594c0 msgr2=0x7f8e940cb170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.462+0000 7f8e93400640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e941594c0 0x7f8e940cb170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.463+0000 7f8e93400640 1 -- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e740036c0 msgr2=0x7f8e940ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.463+0000 7f8e93400640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e740036c0 0x7f8e940ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.463+0000 7f8e93400640 1 -- 172.21.15.47:0/2589814321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e94146240 con 0x7f8e94159ed0 2024-04-24T22:41:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.463+0000 7f8e92a00640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e740036c0 0x7f8e940ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:27.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.463+0000 7f8e93400640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159ed0 0x7f8e94146540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e9406f350 tx=0x7f8e8c014420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:27.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.463+0000 7f8e8aa00640 1 -- 172.21.15.47:0/2589814321 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c005d50 con 0x7f8e94159ed0 2024-04-24T22:41:27.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.463+0000 7f8e8aa00640 1 -- 172.21.15.47:0/2589814321 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e8c014b00 con 0x7f8e94159ed0 2024-04-24T22:41:27.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.463+0000 7f8e93e00640 1 -- 172.21.15.47:0/2589814321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e94144810 con 0x7f8e94159ed0 2024-04-24T22:41:27.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.463+0000 7f8e8aa00640 1 -- 172.21.15.47:0/2589814321 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c012110 con 0x7f8e94159ed0 2024-04-24T22:41:27.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.463+0000 7f8e93e00640 1 -- 172.21.15.47:0/2589814321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e94144dd0 con 0x7f8e94159ed0 2024-04-24T22:41:27.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.465+0000 7f8e93e00640 1 -- 172.21.15.47:0/2589814321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e94068d50 con 0x7f8e94159ed0 2024-04-24T22:41:27.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.468+0000 7f8e8aa00640 1 -- 172.21.15.47:0/2589814321 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e8c0122b0 con 0x7f8e94159ed0 2024-04-24T22:41:27.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.468+0000 7f8e8aa00640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8e7406cbe0 0x7f8e7406f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.469+0000 7f8e8aa00640 1 -- 172.21.15.47:0/2589814321 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8e8c092df0 con 0x7f8e94159ed0 2024-04-24T22:41:27.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.469+0000 7f8e92a00640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8e7406cbe0 0x7f8e7406f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:27.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.470+0000 7f8e92a00640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8e7406cbe0 0x7f8e7406f0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8e84034000 tx=0x7f8e8402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:27.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.475+0000 7f8e8aa00640 1 -- 172.21.15.47:0/2589814321 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e94146240 con 0x7f8e94159ed0 2024-04-24T22:41:27.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.784+0000 7f8e93e00640 1 -- 172.21.15.47:0/2589814321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8e940c2750 con 0x7f8e94159ed0 2024-04-24T22:41:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.784+0000 7f8e8aa00640 1 -- 172.21.15.47:0/2589814321 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f8e8c061790 con 0x7f8e94159ed0 2024-04-24T22:41:27.786 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":14,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:41:27.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 -- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8e7406cbe0 msgr2=0x7f8e7406f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8e7406cbe0 0x7f8e7406f0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8e84034000 tx=0x7f8e8402f000 comp rx=0 tx=0).stop 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 -- 172.21.15.47:0/2589814321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159ed0 msgr2=0x7f8e94146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159ed0 0x7f8e94146540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8e9406f350 tx=0x7f8e8c014420 comp rx=0 tx=0).stop 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 -- 172.21.15.47:0/2589814321 shutdown_connections 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8e7406cbe0 0x7f8e7406f0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159ed0 0x7f8e94146540 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e941594c0 0x7f8e940cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 --2- 172.21.15.47:0/2589814321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e740036c0 0x7f8e940ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 -- 172.21.15.47:0/2589814321 >> 172.21.15.47:0/2589814321 conn(0x7f8e940cb900 msgr2=0x7f8e940cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 -- 172.21.15.47:0/2589814321 shutdown_connections 2024-04-24T22:41:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.787+0000 7f8e93e00640 1 -- 172.21.15.47:0/2589814321 wait complete. 2024-04-24T22:41:27.813 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-24T22:41:27.813 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-24T22:41:27.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:41:27.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.983+0000 7f49b6c00640 1 Processor -- start 2024-04-24T22:41:27.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.983+0000 7f49b6c00640 1 -- start start 2024-04-24T22:41:27.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.985+0000 7f49b6c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49b0152390 0x7f49b00c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.985+0000 7f49b6c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f49b006df80 0x7f49b00c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.985+0000 7f49b6c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b00c88c0 con 0x7f49b006db70 2024-04-24T22:41:27.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.985+0000 7f49b6c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f49b00c4bf0 con 0x7f49b006df80 2024-04-24T22:41:27.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.985+0000 7f49b6c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f49b015c660 con 0x7f49b0152390 2024-04-24T22:41:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.985+0000 7f49b4e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f49b006db70 0x7f49b015bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:51996/0 (socket says 172.21.15.47:51996) 2024-04-24T22:41:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.985+0000 7f49b4e00640 1 -- 172.21.15.47:0/1627061130 learned_addr learned my addr 172.21.15.47:0/1627061130 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.985+0000 7f49b6200640 1 --2- 172.21.15.47:0/1627061130 >> v2:172.21.15.47:3300/0 conn(0x7f49b006df80 0x7f49b00c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.985+0000 7f49b5800640 1 --2- 172.21.15.47:0/1627061130 >> v2:172.21.15.113:3300/0 conn(0x7f49b0152390 0x7f49b00c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.986+0000 7f49afe00640 1 -- 172.21.15.47:0/1627061130 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 194236886 0 0) 0x7f49b00c88c0 con 0x7f49b006db70 2024-04-24T22:41:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.986+0000 7f49afe00640 1 -- 172.21.15.47:0/1627061130 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4998003520 con 0x7f49b006db70 2024-04-24T22:41:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.986+0000 7f49b5800640 1 -- 172.21.15.47:0/1627061130 >> v2:172.21.15.47:3300/0 conn(0x7f49b006df80 msgr2=0x7f49b00c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.986+0000 7f49b5800640 1 --2- 172.21.15.47:0/1627061130 >> v2:172.21.15.47:3300/0 conn(0x7f49b006df80 0x7f49b00c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.986+0000 7f49b5800640 1 -- 172.21.15.47:0/1627061130 >> v1:172.21.15.113:6789/0 conn(0x7f49b006db70 legacy=0x7f49b015bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.986+0000 7f49b5800640 1 -- 172.21.15.47:0/1627061130 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b0152770 con 0x7f49b0152390 2024-04-24T22:41:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.987+0000 7f49b5800640 1 --2- 172.21.15.47:0/1627061130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b0152390 0x7f49b00c5890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f49b00cb0a0 tx=0x7f49a402a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1857d3d6faca9bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:27.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.988+0000 7f49afe00640 1 -- 172.21.15.47:0/1627061130 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4037070 con 0x7f49b0152390 2024-04-24T22:41:27.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.988+0000 7f49afe00640 1 -- 172.21.15.47:0/1627061130 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a4032440 con 0x7f49b0152390 2024-04-24T22:41:27.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.988+0000 7f49afe00640 1 -- 172.21.15.47:0/1627061130 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4032810 con 0x7f49b0152390 2024-04-24T22:41:27.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.988+0000 7f49b6c00640 1 -- 172.21.15.47:0/1627061130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b0152390 msgr2=0x7f49b00c5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:27.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.988+0000 7f49b6c00640 1 --2- 172.21.15.47:0/1627061130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b0152390 0x7f49b00c5890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f49b00cb0a0 tx=0x7f49a402a1f0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.990+0000 7f49b6c00640 1 -- 172.21.15.47:0/1627061130 shutdown_connections 2024-04-24T22:41:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.990+0000 7f49b6c00640 1 --2- 172.21.15.47:0/1627061130 >> v2:172.21.15.47:3300/0 conn(0x7f49b006df80 0x7f49b00c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.990+0000 7f49b6c00640 1 --2- 172.21.15.47:0/1627061130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b0152390 0x7f49b00c5890 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.990+0000 7f49b6c00640 1 -- 172.21.15.47:0/1627061130 >> 172.21.15.47:0/1627061130 conn(0x7f49b00729f0 msgr2=0x7f49b0072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.990+0000 7f49b6c00640 1 -- 172.21.15.47:0/1627061130 shutdown_connections 2024-04-24T22:41:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.990+0000 7f49b6c00640 1 -- 172.21.15.47:0/1627061130 wait complete. 2024-04-24T22:41:27.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.992+0000 7f49b6c00640 1 Processor -- start 2024-04-24T22:41:27.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.993+0000 7f49b6c00640 1 -- start start 2024-04-24T22:41:27.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.994+0000 7f49b6c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b006db70 0x7f49b01f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.994+0000 7f49b6c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b006df80 0x7f49b01535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.994+0000 7f49b6c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b0152390 0x7f49b0154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.994+0000 7f49b6c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f49b015b750 con 0x7f49b006df80 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.994+0000 7f49b6c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49b015bac0 con 0x7f49b006db70 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.994+0000 7f49b6c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f49b00c8e60 con 0x7f49b0152390 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.994+0000 7f49b6200640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b0152390 msgr2=0x7f49b0154e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.994+0000 7f49b6200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b0152390 0x7f49b0154e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.994+0000 7f49b5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b006db70 0x7f49b01f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.994+0000 7f49b5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b006db70 0x7f49b01f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:46150/0 (socket says 172.21.15.47:46150) 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.995+0000 7f49b5800640 1 -- 172.21.15.47:0/1291607628 learned_addr learned my addr 172.21.15.47:0/1291607628 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.995+0000 7f49b4e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b006df80 0x7f49b01535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.995+0000 7f49b5800640 1 -- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b0152390 msgr2=0x7f49b0154e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.995+0000 7f49b5800640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49b0152390 0x7f49b0154e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.995+0000 7f49b5800640 1 -- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b006df80 msgr2=0x7f49b01535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.995+0000 7f49b5800640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b006df80 0x7f49b01535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.995+0000 7f49b5800640 1 -- 172.21.15.47:0/1291607628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b01543a0 con 0x7f49b006db70 2024-04-24T22:41:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.995+0000 7f49b5800640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b006db70 0x7f49b01f0050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f49a4000c00 tx=0x7f49a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:27.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.996+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4041020 con 0x7f49b006db70 2024-04-24T22:41:27.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.996+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a403d700 con 0x7f49b006db70 2024-04-24T22:41:27.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.996+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4037040 con 0x7f49b006db70 2024-04-24T22:41:27.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.996+0000 7f49b6c00640 1 -- 172.21.15.47:0/1291607628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b01f0590 con 0x7f49b006db70 2024-04-24T22:41:27.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.996+0000 7f49b6c00640 1 -- 172.21.15.47:0/1291607628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b01f0b00 con 0x7f49b006db70 2024-04-24T22:41:27.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.997+0000 7f49b6c00640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b00cb960 0x7f49b014f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:27.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.997+0000 7f49b6c00640 1 -- 172.21.15.47:0/1291607628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f49b00c6320 con 0x7f49b00cb960 2024-04-24T22:41:27.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.997+0000 7f49b4e00640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b00cb960 0x7f49b014f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:27.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.998+0000 7f49b4e00640 1 -- 172.21.15.47:0/1291607628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49b004f5e0 con 0x7f49b00cb960 2024-04-24T22:41:27.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.998+0000 7f49b4e00640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b00cb960 0x7f49b014f2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49b00c44b0 tx=0x7f499c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:27.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.998+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c00f5e0 con 0x7f49b00cb960 2024-04-24T22:41:28.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.998+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f499c018070 con 0x7f49b00cb960 2024-04-24T22:41:28.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.998+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b00cb960 msgr2=0x7f49b014f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.998+0000 7f49ad600640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b00cb960 0x7f49b014f2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49b00c44b0 tx=0x7f499c001970 comp rx=0 tx=0).stop 2024-04-24T22:41:28.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.998+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b00cb960 msgr2=0x7f49b014f2a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:28.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.998+0000 7f49ad600640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b00cb960 0x7f49b014f2a0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49b00c44b0 tx=0x7f499c001970 comp rx=0 tx=0).stop 2024-04-24T22:41:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:27.999+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49a4044020 con 0x7f49b006db70 2024-04-24T22:41:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.001+0000 7f49ad600640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f499006b910 0x7f499006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.001+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f49a40ab540 con 0x7f49b006db70 2024-04-24T22:41:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.001+0000 7f49b4e00640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f499006b910 0x7f499006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.002+0000 7f49b4e00640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f499006b910 0x7f499006ddd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f49b00c44b0 tx=0x7f499c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:28.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.007+0000 7f49b6c00640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0153e50 0x7f49b014f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.007+0000 7f49b6c00640 1 -- 172.21.15.47:0/1291607628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f49b014fd20 con 0x7f49b0153e50 2024-04-24T22:41:28.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.007+0000 7f49b6200640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0153e50 0x7f49b014f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.007+0000 7f49b6200640 1 -- 172.21.15.47:0/1291607628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f49b00c49b0 con 0x7f49b0153e50 2024-04-24T22:41:28.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.008+0000 7f49b6200640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0153e50 0x7f49b014f7e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49b015ba70 tx=0x7f49a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:28.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.008+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8018030 con 0x7f49b0153e50 2024-04-24T22:41:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.008+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5028 (secure 0 0 0) 0x7f49a800cd20 con 0x7f49b0153e50 2024-04-24T22:41:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.008+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0153e50 msgr2=0x7f49b014f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.008+0000 7f49ad600640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0153e50 0x7f49b014f7e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49b015ba70 tx=0x7f49a800c490 comp rx=0 tx=0).stop 2024-04-24T22:41:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.008+0000 7f49ad600640 1 -- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0153e50 msgr2=0x7f49b014f7e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.008+0000 7f49ad600640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49b0153e50 0x7f49b014f7e0 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49b015ba70 tx=0x7f49a800c490 comp rx=0 tx=0).stop 2024-04-24T22:41:28.010 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:28.010 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 14, 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 28045, 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: 1 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:28.011 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:28.012 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:28.013 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:28.014 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:28.015 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:28.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.018 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.020 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.021 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.023 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:28.023 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:28.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.009+0000 7f49b6c00640 1 -- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f499006b910 msgr2=0x7f499006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.009+0000 7f49b6c00640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f499006b910 0x7f499006ddd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f49b00c44b0 tx=0x7f499c009000 comp rx=0 tx=0).stop 2024-04-24T22:41:28.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.009+0000 7f49b6c00640 1 -- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b006db70 msgr2=0x7f49b01f0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.009+0000 7f49b6c00640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b006db70 0x7f49b01f0050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f49a4000c00 tx=0x7f49a40046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.009+0000 7f49b6200640 1 -- 172.21.15.47:0/1291607628 reap_dead start 2024-04-24T22:41:28.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.009+0000 7f49b6c00640 1 -- 172.21.15.47:0/1291607628 shutdown_connections 2024-04-24T22:41:28.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.009+0000 7f49b6c00640 1 --2- 172.21.15.47:0/1291607628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49b006db70 0x7f49b01f0050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.009+0000 7f49b6c00640 1 -- 172.21.15.47:0/1291607628 >> 172.21.15.47:0/1291607628 conn(0x7f49b00729f0 msgr2=0x7f49b0156ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:28.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.009+0000 7f49b6c00640 1 -- 172.21.15.47:0/1291607628 shutdown_connections 2024-04-24T22:41:28.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.009+0000 7f49b6c00640 1 -- 172.21.15.47:0/1291607628 wait complete. 2024-04-24T22:41:28.028 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-24T22:41:28.028 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-24T22:41:28.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:41:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.206+0000 7f1385200640 1 Processor -- start 2024-04-24T22:41:28.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.207+0000 7f1385200640 1 -- start start 2024-04-24T22:41:28.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.208+0000 7f1385200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1380072c60 0x7f13800c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.208+0000 7f1385200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1380073040 0x7f13801411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.208+0000 7f1385200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138015c3d0 con 0x7f138006bb90 2024-04-24T22:41:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.208+0000 7f1385200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1380158bf0 con 0x7f1380073040 2024-04-24T22:41:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.208+0000 7f1385200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f138015c6b0 con 0x7f1380072c60 2024-04-24T22:41:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.208+0000 7f137f400640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1380072c60 msgr2=0x7f13800c2d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:41:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.208+0000 7f137f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1380072c60 0x7f13800c2d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.208+0000 7f137ea00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1380073040 0x7f13801411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.208+0000 7f137ea00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1380073040 0x7f13801411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38586/0 (socket says 172.21.15.47:38586) 2024-04-24T22:41:28.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.208+0000 7f137ea00640 1 -- 172.21.15.47:0/513568291 learned_addr learned my addr 172.21.15.47:0/513568291 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.209+0000 7f137ea00640 1 -- 172.21.15.47:0/513568291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1380072c60 msgr2=0x7f13800c2d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:41:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.209+0000 7f137ea00640 1 --2- 172.21.15.47:0/513568291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1380072c60 0x7f13800c2d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.209+0000 7f137ea00640 1 -- 172.21.15.47:0/513568291 >> v1:172.21.15.47:6789/0 conn(0x7f138006bb90 legacy=0x7f138015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.209+0000 7f137ea00640 1 -- 172.21.15.47:0/513568291 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13801585a0 con 0x7f1380073040 2024-04-24T22:41:28.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.210+0000 7f137ea00640 1 --2- 172.21.15.47:0/513568291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1380073040 0x7f13801411f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f136c009870 tx=0x7f136c02a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b58f0664ae0f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.210+0000 7f137e000640 1 -- 172.21.15.47:0/513568291 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c037070 con 0x7f1380073040 2024-04-24T22:41:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.210+0000 7f137e000640 1 -- 172.21.15.47:0/513568291 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c032640 con 0x7f1380073040 2024-04-24T22:41:28.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.210+0000 7f137e000640 1 -- 172.21.15.47:0/513568291 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c02ae60 con 0x7f1380073040 2024-04-24T22:41:28.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.211+0000 7f1385200640 1 -- 172.21.15.47:0/513568291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1380073040 msgr2=0x7f13801411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.211+0000 7f1385200640 1 --2- 172.21.15.47:0/513568291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1380073040 0x7f13801411f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f136c009870 tx=0x7f136c02a310 comp rx=0 tx=0).stop 2024-04-24T22:41:28.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.211+0000 7f1385200640 1 -- 172.21.15.47:0/513568291 shutdown_connections 2024-04-24T22:41:28.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.211+0000 7f1385200640 1 --2- 172.21.15.47:0/513568291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1380073040 0x7f13801411f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.211+0000 7f1385200640 1 --2- 172.21.15.47:0/513568291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1380072c60 0x7f13800c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.211+0000 7f1385200640 1 -- 172.21.15.47:0/513568291 >> 172.21.15.47:0/513568291 conn(0x7f13800cb900 msgr2=0x7f13800cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:28.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.211+0000 7f1385200640 1 -- 172.21.15.47:0/513568291 shutdown_connections 2024-04-24T22:41:28.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.211+0000 7f1385200640 1 -- 172.21.15.47:0/513568291 wait complete. 2024-04-24T22:41:28.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.213+0000 7f1385200640 1 Processor -- start 2024-04-24T22:41:28.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.213+0000 7f1385200640 1 -- start start 2024-04-24T22:41:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.214+0000 7f1385200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f138006bb90 0x7f13800c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.214+0000 7f1385200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1380072c60 0x7f13800c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.214+0000 7f1385200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380073040 0x7f138013da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.214+0000 7f1385200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f13800c32a0 con 0x7f1380072c60 2024-04-24T22:41:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.214+0000 7f1385200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1380158270 con 0x7f1380073040 2024-04-24T22:41:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.214+0000 7f1385200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1380159080 con 0x7f138006bb90 2024-04-24T22:41:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.214+0000 7f137f400640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f138006bb90 msgr2=0x7f13800c6c50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:41:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.214+0000 7f137f400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f138006bb90 0x7f13800c6c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.214+0000 7f137ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1380072c60 0x7f13800c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.214+0000 7f137fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380073040 0x7f138013da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.215+0000 7f137ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1380072c60 0x7f13800c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38598/0 (socket says 172.21.15.47:38598) 2024-04-24T22:41:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.215+0000 7f137ea00640 1 -- 172.21.15.47:0/455655173 learned_addr learned my addr 172.21.15.47:0/455655173 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.215+0000 7f137ea00640 1 -- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f138006bb90 msgr2=0x7f13800c6c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.215+0000 7f137ea00640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f138006bb90 0x7f13800c6c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.215+0000 7f137ea00640 1 -- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380073040 msgr2=0x7f138013da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.215+0000 7f137ea00640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380073040 0x7f138013da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.215+0000 7f137ea00640 1 -- 172.21.15.47:0/455655173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13800c7190 con 0x7f1380072c60 2024-04-24T22:41:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.215+0000 7f137fe00640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380073040 0x7f138013da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.215+0000 7f137ea00640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1380072c60 0x7f13800c8a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f136c009870 tx=0x7f136c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.216+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c041070 con 0x7f1380072c60 2024-04-24T22:41:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.216+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c03c870 con 0x7f1380072c60 2024-04-24T22:41:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.216+0000 7f1385200640 1 -- 172.21.15.47:0/455655173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138015af30 con 0x7f1380072c60 2024-04-24T22:41:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.216+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c037040 con 0x7f1380072c60 2024-04-24T22:41:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.216+0000 7f1385200640 1 -- 172.21.15.47:0/455655173 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13801e4270 con 0x7f1380072c60 2024-04-24T22:41:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.217+0000 7f1385200640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138013e4e0 0x7f138013df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.217+0000 7f1385200640 1 -- 172.21.15.47:0/455655173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f13800c84f0 con 0x7f138013e4e0 2024-04-24T22:41:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.218+0000 7f137f400640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138013e4e0 0x7f138013df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.218+0000 7f137f400640 1 -- 172.21.15.47:0/455655173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f138015ace0 con 0x7f138013e4e0 2024-04-24T22:41:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.218+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f136c044070 con 0x7f1380072c60 2024-04-24T22:41:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.218+0000 7f137f400640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138013e4e0 0x7f138013df70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1370003930 tx=0x7f137000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.219+0000 7f1377400640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f136006bde0 0x7f136006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.219+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f136c0ab870 con 0x7f1380072c60 2024-04-24T22:41:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.219+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370003e10 con 0x7f138013e4e0 2024-04-24T22:41:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.219+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1370015070 con 0x7f138013e4e0 2024-04-24T22:41:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.219+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138013e4e0 msgr2=0x7f138013df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.219+0000 7f1377400640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138013e4e0 0x7f138013df70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1370003930 tx=0x7f137000b8c0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.219+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138013e4e0 msgr2=0x7f138013df70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.219+0000 7f1377400640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138013e4e0 0x7f138013df70 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1370003930 tx=0x7f137000b8c0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.219+0000 7f137fe00640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f136006bde0 0x7f136006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.220+0000 7f137fe00640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f136006bde0 0x7f136006e2a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f13801f14e0 tx=0x7f1378009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:28.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.227+0000 7f1385200640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380072240 0x7f13800c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:28.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.227+0000 7f1385200640 1 -- 172.21.15.47:0/455655173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1380068d50 con 0x7f1380072240 2024-04-24T22:41:28.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.227+0000 7f137f400640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380072240 0x7f13800c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:28.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.228+0000 7f137f400640 1 -- 172.21.15.47:0/455655173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f138013f190 con 0x7f1380072240 2024-04-24T22:41:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.228+0000 7f137f400640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380072240 0x7f13800c6590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1370003930 tx=0x7f1370005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.228+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370005550 con 0x7f1380072240 2024-04-24T22:41:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.229+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5054 (secure 0 0 0) 0x7f1370005550 con 0x7f1380072240 2024-04-24T22:41:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.229+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380072240 msgr2=0x7f13800c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.229+0000 7f1377400640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380072240 0x7f13800c6590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1370003930 tx=0x7f1370005a20 comp rx=0 tx=0).stop 2024-04-24T22:41:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.229+0000 7f1377400640 1 -- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380072240 msgr2=0x7f13800c6590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:28.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.229+0000 7f1377400640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1380072240 0x7f13800c6590 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1370003930 tx=0x7f1370005a20 comp rx=0 tx=0).stop 2024-04-24T22:41:28.232 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:28.232 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:28.232 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:28.232 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:41:28.232 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 14, 2024-04-24T22:41:28.232 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 129796, 2024-04-24T22:41:28.232 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: 1 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:28.233 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:28.234 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:28.235 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:28.236 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.237 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.238 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:28.240 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:28.241 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.242 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:41:28.243 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:41:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7 2024-04-24T22:41:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:28.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:28.246 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:28.246 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:28.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.229+0000 7f1385200640 1 -- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f136006bde0 msgr2=0x7f136006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.229+0000 7f1385200640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f136006bde0 0x7f136006e2a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f13801f14e0 tx=0x7f1378009040 comp rx=0 tx=0).stop 2024-04-24T22:41:28.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.229+0000 7f1385200640 1 -- 172.21.15.47:0/455655173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1380072c60 msgr2=0x7f13800c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:28.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.229+0000 7f1385200640 1 --2- 172.21.15.47:0/455655173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1380072c60 0x7f13800c8a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f136c009870 tx=0x7f136c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:28.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.229+0000 7f137fe00640 1 -- 172.21.15.47:0/455655173 reap_dead start 2024-04-24T22:41:28.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.230+0000 7f137fe00640 1 -- 172.21.15.47:0/455655173 reap_dead start 2024-04-24T22:41:28.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.230+0000 7f1385200640 1 -- 172.21.15.47:0/455655173 shutdown_connections 2024-04-24T22:41:28.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.230+0000 7f1385200640 1 -- 172.21.15.47:0/455655173 >> 172.21.15.47:0/455655173 conn(0x7f13800cb900 msgr2=0x7f13801560c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.230+0000 7f1385200640 1 -- 172.21.15.47:0/455655173 shutdown_connections 2024-04-24T22:41:28.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:28.230+0000 7f1385200640 1 -- 172.21.15.47:0/455655173 wait complete. 2024-04-24T22:41:28.249 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-24T22:41:28.249 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-24T22:41:40.313 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:41:40.322 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:41:43.138 INFO:tasks.workunit.client.0.smithi047.stderr:Updating files: 61% (7727/12492) Updating files: 62% (7746/12492) Updating files: 63% (7870/12492) Updating files: 64% (7995/12492) Updating files: 65% (8120/12492) Updating files: 66% (8245/12492) Updating files: 67% (8370/12492) Updating files: 68% (8495/12492) Updating files: 69% (8620/12492) Updating files: 70% (8745/12492) Updating files: 71% (8870/12492) Updating files: 72% (8995/12492) Updating files: 73% (9120/12492) Updating files: 74% (9245/12492) Updating files: 75% (9369/12492) Updating files: 76% (9494/12492) Updating files: 77% (9619/12492) Updating files: 78% (9744/12492) Updating files: 79% (9869/12492) Updating files: 80% (9994/12492) Updating files: 81% (10119/12492) Updating files: 82% (10244/12492) Updating files: 83% (10369/12492) Updating files: 84% (10494/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 98% (12337/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-24T22:41:43.540 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T22:41:43.540 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-24T22:41:43.540 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T22:41:43.540 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T22:41:43.540 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T22:41:43.540 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-24T22:41:43.540 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T22:41:43.540 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T22:41:43.540 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-24T22:41:43.540 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-04-24T22:41:43.541 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-24T22:41:43.541 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-04-24T22:41:43.541 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-24T22:41:43.541 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-04-24T22:41:43.541 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-24T22:41:43.541 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T22:41:43.541 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-04-24T22:41:43.541 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T22:41:43.549 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T22:41:43.613 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T22:41:43.623 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T22:41:43.623 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T22:41:44.379 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T22:41:44.486 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T22:41:44.576 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T22:41:44.582 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T22:41:44.582 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T22:41:44.669 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T22:41:44.681 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:41:44.681 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T22:41:44.736 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-04-24T22:41:44.738 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-04-24T22:41:44.738 DEBUG:teuthology.orchestra.run.smithi047:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-04-24T22:41:44.813 INFO:tasks.workunit.client.0.smithi047.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: Iozone: Performance Test of File I/O 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: Version $Revision: 3.494 $ 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: Compiled for 64 bit mode. 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: Build: linux-AMD64 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-24T22:41:44.916 INFO:tasks.workunit.client.0.smithi047.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Run began: Wed Apr 24 22:41:44 2024 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Include close in write timing 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Include fsync in write timing 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: File size set to 1048576 kB 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Record Size 16 kB 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Output is in kBytes/sec 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Time Resolution = 0.000001 seconds. 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Processor cache size set to 1024 kBytes. 2024-04-24T22:41:44.917 INFO:tasks.workunit.client.0.smithi047.stdout: Processor cache line size set to 32 bytes. 2024-04-24T22:41:44.918 INFO:tasks.workunit.client.0.smithi047.stdout: File stride size set to 17 * record size. 2024-04-24T22:41:44.918 INFO:tasks.workunit.client.0.smithi047.stdout: Throughput test with 1 process 2024-04-24T22:41:44.918 INFO:tasks.workunit.client.0.smithi047.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-04-24T22:41:48.250 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-24T22:41:48.250 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-24T22:41:48.250 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T22:41:48.251 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-24T22:41:48.253 INFO:tasks.ceph.mon.c:Started 2024-04-24T22:41:48.253 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-24T22:41:48.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:41:48.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.514+0000 7fac8ba00640 1 Processor -- start 2024-04-24T22:41:48.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.515+0000 7fac8ba00640 1 -- start start 2024-04-24T22:41:48.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.517+0000 7fac8ba00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fac840c7df0 0x7fac840cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.517+0000 7fac8ba00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac84156d00 con 0x7fac84150c10 2024-04-24T22:41:48.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.517+0000 7fac8ba00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac84156fe0 con 0x7fac840cc6e0 2024-04-24T22:41:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.517+0000 7fac8ba00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fac841572c0 con 0x7fac840c7df0 2024-04-24T22:41:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.517+0000 7fac89000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fac84150c10 0x7fac840c8360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37734/0 (socket says 172.21.15.47:37734) 2024-04-24T22:41:48.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.518+0000 7fac89000640 1 -- 172.21.15.47:0/3968277883 learned_addr learned my addr 172.21.15.47:0/3968277883 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.518+0000 7fac83e00640 1 --2- 172.21.15.47:0/3968277883 >> v2:172.21.15.47:3300/0 conn(0x7fac840c7df0 0x7fac840cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.519+0000 7fac83400640 1 -- 172.21.15.47:0/3968277883 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009596218 0 0) 0x7fac84156fe0 con 0x7fac840cc6e0 2024-04-24T22:41:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.519+0000 7fac83400640 1 -- 172.21.15.47:0/3968277883 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac6c003440 con 0x7fac840cc6e0 2024-04-24T22:41:48.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.520+0000 7fac83400640 1 -- 172.21.15.47:0/3968277883 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1864496773 0 0) 0x7fac84156d00 con 0x7fac84150c10 2024-04-24T22:41:48.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.520+0000 7fac83400640 1 -- 172.21.15.47:0/3968277883 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac84156fe0 con 0x7fac84150c10 2024-04-24T22:41:48.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.521+0000 7fac83e00640 1 -- 172.21.15.47:0/3968277883 >> v1:172.21.15.113:6789/0 conn(0x7fac840cc6e0 legacy=0x7fac840720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.521+0000 7fac83e00640 1 -- 172.21.15.47:0/3968277883 >> v1:172.21.15.47:6789/0 conn(0x7fac84150c10 legacy=0x7fac840c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.521+0000 7fac83e00640 1 -- 172.21.15.47:0/3968277883 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac841575a0 con 0x7fac840c7df0 2024-04-24T22:41:48.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.523+0000 7fac83e00640 1 --2- 172.21.15.47:0/3968277883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac840c7df0 0x7fac840cc1a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fac700097a0 tx=0x7fac7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a88c372c7c0929 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.525+0000 7fac83400640 1 -- 172.21.15.47:0/3968277883 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70037070 con 0x7fac840c7df0 2024-04-24T22:41:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.525+0000 7fac83400640 1 -- 172.21.15.47:0/3968277883 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac70032440 con 0x7fac840c7df0 2024-04-24T22:41:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.526+0000 7fac83400640 1 -- 172.21.15.47:0/3968277883 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70037070 con 0x7fac840c7df0 2024-04-24T22:41:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.526+0000 7fac8ba00640 1 -- 172.21.15.47:0/3968277883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac840c7df0 msgr2=0x7fac840cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.526+0000 7fac8ba00640 1 --2- 172.21.15.47:0/3968277883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac840c7df0 0x7fac840cc1a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fac700097a0 tx=0x7fac7002a1e0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.533+0000 7fac8ba00640 1 -- 172.21.15.47:0/3968277883 shutdown_connections 2024-04-24T22:41:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.533+0000 7fac8ba00640 1 --2- 172.21.15.47:0/3968277883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac840c7df0 0x7fac840cc1a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.533+0000 7fac8ba00640 1 -- 172.21.15.47:0/3968277883 >> 172.21.15.47:0/3968277883 conn(0x7fac8406b160 msgr2=0x7fac8406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.533+0000 7fac8ba00640 1 -- 172.21.15.47:0/3968277883 shutdown_connections 2024-04-24T22:41:48.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.533+0000 7fac8ba00640 1 -- 172.21.15.47:0/3968277883 wait complete. 2024-04-24T22:41:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.536+0000 7fac8ba00640 1 Processor -- start 2024-04-24T22:41:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.536+0000 7fac8ba00640 1 -- start start 2024-04-24T22:41:48.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.538+0000 7fac8ba00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac840c7df0 0x7fac841eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.538+0000 7fac8ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac840cc6e0 0x7fac84149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.538+0000 7fac8ba00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac84150c10 0x7fac8413d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.538+0000 7fac8ba00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fac841569d0 con 0x7fac840c7df0 2024-04-24T22:41:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.538+0000 7fac8ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fac84071ba0 con 0x7fac840cc6e0 2024-04-24T22:41:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.538+0000 7fac8ba00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fac84150ff0 con 0x7fac84150c10 2024-04-24T22:41:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.539+0000 7fac89000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac840c7df0 0x7fac841eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.539+0000 7fac83e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac840cc6e0 0x7fac84149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.539+0000 7fac83e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac840cc6e0 0x7fac84149b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:50144/0 (socket says 172.21.15.47:50144) 2024-04-24T22:41:48.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.539+0000 7fac83e00640 1 -- 172.21.15.47:0/20528958 learned_addr learned my addr 172.21.15.47:0/20528958 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.539+0000 7fac89a00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac84150c10 0x7fac8413d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.540+0000 7fac83e00640 1 -- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac84150c10 msgr2=0x7fac8413d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.540+0000 7fac83e00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac84150c10 0x7fac8413d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.540+0000 7fac83e00640 1 -- 172.21.15.47:0/20528958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac840c7df0 msgr2=0x7fac841eccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.540+0000 7fac83e00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac840c7df0 0x7fac841eccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.540+0000 7fac83e00640 1 -- 172.21.15.47:0/20528958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac841517b0 con 0x7fac840cc6e0 2024-04-24T22:41:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.543+0000 7fac83e00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac840cc6e0 0x7fac84149b30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac7002a7d0 tx=0x7fac700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.543+0000 7fac80c00640 1 -- 172.21.15.47:0/20528958 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70041020 con 0x7fac840cc6e0 2024-04-24T22:41:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.545+0000 7fac80c00640 1 -- 172.21.15.47:0/20528958 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7003d620 con 0x7fac840cc6e0 2024-04-24T22:41:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.545+0000 7fac80c00640 1 -- 172.21.15.47:0/20528958 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70037040 con 0x7fac840cc6e0 2024-04-24T22:41:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.545+0000 7fac8ba00640 1 -- 172.21.15.47:0/20528958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac84151a50 con 0x7fac840cc6e0 2024-04-24T22:41:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.545+0000 7fac8ba00640 1 -- 172.21.15.47:0/20528958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac841ec4d0 con 0x7fac840cc6e0 2024-04-24T22:41:48.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.547+0000 7fac8ba00640 1 -- 172.21.15.47:0/20528958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac84073b90 con 0x7fac840cc6e0 2024-04-24T22:41:48.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.548+0000 7fac80c00640 1 -- 172.21.15.47:0/20528958 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac70044020 con 0x7fac840cc6e0 2024-04-24T22:41:48.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.549+0000 7fac80c00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fac6406bf00 0x7fac6406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:48.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.549+0000 7fac80c00640 1 -- 172.21.15.47:0/20528958 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fac700ab610 con 0x7fac840cc6e0 2024-04-24T22:41:48.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.549+0000 7fac89000640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fac6406bf00 0x7fac6406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:48.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.550+0000 7fac89000640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fac6406bf00 0x7fac6406e3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac78007b40 tx=0x7fac78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.555+0000 7fac80c00640 1 -- 172.21.15.47:0/20528958 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fac7004b030 con 0x7fac840cc6e0 2024-04-24T22:41:48.637 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:41:48.637 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 initial writers = 282170.56 kB/sec 2024-04-24T22:41:48.637 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 initial writers = 282117.65 kB/sec 2024-04-24T22:41:48.637 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 282170.56 kB/sec 2024-04-24T22:41:48.637 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 282170.56 kB/sec 2024-04-24T22:41:48.637 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 282170.56 kB/sec 2024-04-24T22:41:48.637 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 1048576.00 kB 2024-04-24T22:41:48.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.892+0000 7fac8ba00640 1 -- 172.21.15.47:0/20528958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fac8406f7d0 con 0x7fac840cc6e0 2024-04-24T22:41:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.892+0000 7fac80c00640 1 -- 172.21.15.47:0/20528958 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1559 (secure 0 0 0) 0x7fac70079f30 con 0x7fac840cc6e0 2024-04-24T22:41:48.895 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":14,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:41:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 -- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fac6406bf00 msgr2=0x7fac6406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fac6406bf00 0x7fac6406e3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac78007b40 tx=0x7fac78009040 comp rx=0 tx=0).stop 2024-04-24T22:41:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 -- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac840cc6e0 msgr2=0x7fac84149b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac840cc6e0 0x7fac84149b30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac7002a7d0 tx=0x7fac700046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 -- 172.21.15.47:0/20528958 shutdown_connections 2024-04-24T22:41:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fac6406bf00 0x7fac6406e3c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac84150c10 0x7fac8413d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac840cc6e0 0x7fac84149b30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 --2- 172.21.15.47:0/20528958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac840c7df0 0x7fac841eccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 -- 172.21.15.47:0/20528958 >> 172.21.15.47:0/20528958 conn(0x7fac8406b160 msgr2=0x7fac84072c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 -- 172.21.15.47:0/20528958 shutdown_connections 2024-04-24T22:41:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:48.895+0000 7fac8ba00640 1 -- 172.21.15.47:0/20528958 wait complete. 2024-04-24T22:41:51.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:41:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.135+0000 7f9522800640 1 Processor -- start 2024-04-24T22:41:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.136+0000 7f9522800640 1 -- start start 2024-04-24T22:41:52.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.137+0000 7f9522800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:52.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.137+0000 7f9522800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f951c15ab10 con 0x7f951c154960 2024-04-24T22:41:52.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.138+0000 7f9522800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f951c15adf0 con 0x7f951c0c8ad0 2024-04-24T22:41:52.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.138+0000 7f9522800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f951c15b0d0 con 0x7f951c15a680 2024-04-24T22:41:52.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.138+0000 7f951b400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f951c154960 0x7f951c15e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37746/0 (socket says 172.21.15.47:37746) 2024-04-24T22:41:52.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.138+0000 7f951b400640 1 -- 172.21.15.47:0/703072291 learned_addr learned my addr 172.21.15.47:0/703072291 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:52.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.138+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:52.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.138+0000 7f9513e00640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 msgr2=0x7f951c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:41:52.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.138+0000 7f9513e00640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 msgr2=0x7f951c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:52.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.138+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:52.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.138+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.339+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.339+0000 7f9513e00640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 msgr2=0x7f951c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:41:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.339+0000 7f9513e00640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 msgr2=0x7f951c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.339+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.340+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:41:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.741+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.742+0000 7f9513e00640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 msgr2=0x7f951c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:41:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.742+0000 7f9513e00640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 msgr2=0x7f951c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:52.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.742+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:52.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:52.742+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:41:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.543+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.544+0000 7f9513e00640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 msgr2=0x7f951c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:41:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.544+0000 7f9513e00640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 msgr2=0x7f951c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.544+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.544+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:41:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180555879 0 0) 0x7f951c15adf0 con 0x7f951c0c8ad0 2024-04-24T22:41:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9500003520 con 0x7f951c0c8ad0 2024-04-24T22:41:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4137359491 0 0) 0x7f9500003520 con 0x7f951c0c8ad0 2024-04-24T22:41:53.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 msgr2=0x7f951c0c8590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:41:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15a680 0x7f951c0c8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 >> v1:172.21.15.47:6789/0 conn(0x7f951c154960 legacy=0x7f951c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c15b3b0 con 0x7f951c0c8ad0 2024-04-24T22:41:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 567141988 0 0) 0x7f95140019f0 con 0x7f951c0c8ad0 2024-04-24T22:41:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 >> v1:172.21.15.113:6789/0 conn(0x7f951c0c8ad0 legacy=0x7f951c0c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95000039b0 0x7f9500003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9500004800 0x7f9500004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95000051c0 0x7f950000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:53.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f951c15b0d0 con 0x7f95000051c0 2024-04-24T22:41:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f950000c030 con 0x7f9500004800 2024-04-24T22:41:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.984+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f950000c230 con 0x7f95000039b0 2024-04-24T22:41:53.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.986+0000 7f951b400640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95000039b0 0x7f9500003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.986+0000 7f9513e00640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9500004800 0x7f9500004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.986+0000 7f951b400640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9500004800 msgr2=0x7f9500004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.986+0000 7f951b400640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9500004800 0x7f9500004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.986+0000 7f951b400640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95000051c0 msgr2=0x7f950000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.986+0000 7f951b400640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95000051c0 0x7f950000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.986+0000 7f951b400640 1 -- 172.21.15.47:0/703072291 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c15b3b0 con 0x7f95000039b0 2024-04-24T22:41:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.988+0000 7f951b400640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95000039b0 0x7f9500003dd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f950c001810 tx=0x7f9500005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71275661d6c8e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.988+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c001460 con 0x7f95000039b0 2024-04-24T22:41:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.988+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f950c02e070 con 0x7f95000039b0 2024-04-24T22:41:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.988+0000 7f951aa00640 1 -- 172.21.15.47:0/703072291 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c02ae00 con 0x7f95000039b0 2024-04-24T22:41:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.990+0000 7f9522800640 1 -- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95000039b0 msgr2=0x7f9500003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.990+0000 7f9522800640 1 --2- 172.21.15.47:0/703072291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95000039b0 0x7f9500003dd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f950c001810 tx=0x7f9500005830 comp rx=0 tx=0).stop 2024-04-24T22:41:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.990+0000 7f951be00640 1 -- 172.21.15.47:0/703072291 reap_dead start 2024-04-24T22:41:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.990+0000 7f9522800640 1 -- 172.21.15.47:0/703072291 shutdown_connections 2024-04-24T22:41:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.990+0000 7f9522800640 1 -- 172.21.15.47:0/703072291 >> 172.21.15.47:0/703072291 conn(0x7f951c06e580 msgr2=0x7f951c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.990+0000 7f9522800640 1 -- 172.21.15.47:0/703072291 shutdown_connections 2024-04-24T22:41:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.991+0000 7f9522800640 1 -- 172.21.15.47:0/703072291 wait complete. 2024-04-24T22:41:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.993+0000 7f9522800640 1 Processor -- start 2024-04-24T22:41:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.994+0000 7f9522800640 1 -- start start 2024-04-24T22:41:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.995+0000 7f9522800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f951c15cae0 0x7f951c15d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.995+0000 7f9522800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15c210 0x7f951c1f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.995+0000 7f9522800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c15cec0 0x7f951c1f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.995+0000 7f9522800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f951c15ea00 con 0x7f951c15cae0 2024-04-24T22:41:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.995+0000 7f9522800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f951c154d40 con 0x7f951c15cec0 2024-04-24T22:41:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.995+0000 7f9522800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f951c15eb80 con 0x7f951c15c210 2024-04-24T22:41:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.995+0000 7f9513e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15c210 0x7f951c1f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:53.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.995+0000 7f9513e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15c210 0x7f951c1f4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:55998/0 (socket says 172.21.15.47:55998) 2024-04-24T22:41:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.995+0000 7f9513e00640 1 -- 172.21.15.47:0/2105631999 learned_addr learned my addr 172.21.15.47:0/2105631999 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f951b400640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f951c15cae0 0x7f951c15d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f9513e00640 1 -- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15c210 msgr2=0x7f951c1f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f9513e00640 1 -- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15c210 msgr2=0x7f951c1f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f9513e00640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15c210 0x7f951c1f4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f9513e00640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15c210 0x7f951c1f4990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f951be00640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c15cec0 0x7f951c1f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f951b400640 1 -- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15c210 msgr2=0x7f951c1f4990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f951b400640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15c210 0x7f951c1f4990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f951b400640 1 -- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c15cec0 msgr2=0x7f951c1f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f951b400640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c15cec0 0x7f951c1f4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f951b400640 1 -- 172.21.15.47:0/2105631999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c1f4490 con 0x7f951c15cae0 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.996+0000 7f951be00640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c15cec0 0x7f951c1f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:41:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.997+0000 7f951b400640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f951c15cae0 0x7f951c15d650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f950c005000 tx=0x7f950c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.997+0000 7f9513400640 1 -- 172.21.15.47:0/2105631999 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c043070 con 0x7f951c15cae0 2024-04-24T22:41:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.997+0000 7f9513400640 1 -- 172.21.15.47:0/2105631999 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f950c004c30 con 0x7f951c15cae0 2024-04-24T22:41:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.997+0000 7f9513400640 1 -- 172.21.15.47:0/2105631999 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c007070 con 0x7f951c15cae0 2024-04-24T22:41:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.997+0000 7f9522800640 1 -- 172.21.15.47:0/2105631999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951c152670 con 0x7f951c15cae0 2024-04-24T22:41:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.997+0000 7f9522800640 1 -- 172.21.15.47:0/2105631999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f951c152c30 con 0x7f951c15cae0 2024-04-24T22:41:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:53.999+0000 7f9522800640 1 -- 172.21.15.47:0/2105631999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f951c073b90 con 0x7f951c15cae0 2024-04-24T22:41:54.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.000+0000 7f9513400640 1 -- 172.21.15.47:0/2105631999 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f950c030050 con 0x7f951c15cae0 2024-04-24T22:41:54.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.002+0000 7f9513400640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f94f806be80 0x7f94f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:54.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.002+0000 7f9513400640 1 -- 172.21.15.47:0/2105631999 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f950c0aa690 con 0x7f951c15cae0 2024-04-24T22:41:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.002+0000 7f9513e00640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f94f806be80 0x7f94f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.003+0000 7f9513e00640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f94f806be80 0x7f94f806e340 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f951c072110 tx=0x7f9504008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.009+0000 7f9513400640 1 -- 172.21.15.47:0/2105631999 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f950c078f10 con 0x7f951c15cae0 2024-04-24T22:41:54.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.456+0000 7f9522800640 1 -- 172.21.15.47:0/2105631999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f951c0cc4c0 con 0x7f951c15cae0 2024-04-24T22:41:54.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.457+0000 7f9513400640 1 -- 172.21.15.47:0/2105631999 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f950c078810 con 0x7f951c15cae0 2024-04-24T22:41:54.461 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:41:54.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.461+0000 7f9522800640 1 -- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f94f806be80 msgr2=0x7f94f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:54.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.461+0000 7f9522800640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f94f806be80 0x7f94f806e340 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f951c072110 tx=0x7f9504008040 comp rx=0 tx=0).stop 2024-04-24T22:41:54.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.461+0000 7f9522800640 1 -- 172.21.15.47:0/2105631999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f951c15cae0 msgr2=0x7f951c15d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:54.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.461+0000 7f9522800640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f951c15cae0 0x7f951c15d650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f950c005000 tx=0x7f950c001120 comp rx=0 tx=0).stop 2024-04-24T22:41:54.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.461+0000 7f9522800640 1 -- 172.21.15.47:0/2105631999 shutdown_connections 2024-04-24T22:41:54.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.461+0000 7f9522800640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f94f806be80 0x7f94f806e340 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:54.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.461+0000 7f9522800640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c15cec0 0x7f951c1f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:54.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.461+0000 7f9522800640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f951c15c210 0x7f951c1f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:54.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.461+0000 7f9522800640 1 --2- 172.21.15.47:0/2105631999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f951c15cae0 0x7f951c15d650 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:54.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.461+0000 7f9522800640 1 -- 172.21.15.47:0/2105631999 >> 172.21.15.47:0/2105631999 conn(0x7f951c06e580 msgr2=0x7f951c0b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:54.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.463+0000 7f9522800640 1 -- 172.21.15.47:0/2105631999 shutdown_connections 2024-04-24T22:41:54.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:54.463+0000 7f9522800640 1 -- 172.21.15.47:0/2105631999 wait complete. 2024-04-24T22:41:55.787 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:41:55.787 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 rewriters = 327044.09 kB/sec 2024-04-24T22:41:55.787 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 rewriters = 326946.81 kB/sec 2024-04-24T22:41:55.787 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 327044.09 kB/sec 2024-04-24T22:41:55.787 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 327044.09 kB/sec 2024-04-24T22:41:55.788 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 327044.09 kB/sec 2024-04-24T22:41:55.788 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 1048576.00 kB 2024-04-24T22:41:57.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:41:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.716+0000 7f60e2000640 1 Processor -- start 2024-04-24T22:41:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.717+0000 7f60e2000640 1 -- start start 2024-04-24T22:41:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.718+0000 7f60e2000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc1514f0 0x7f60dc06e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.718+0000 7f60e2000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60dc06e6f0 0x7f60dc0c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.718+0000 7f60e2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60dc0c6850 0x7f60dc0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.718+0000 7f60e2000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f60dc15b470 con 0x7f60dc06e6f0 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.718+0000 7f60e2000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f60dc071570 con 0x7f60dc0c6850 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.718+0000 7f60e2000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc0c6cc0 con 0x7f60dc1514f0 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60dbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60dc06e6f0 0x7f60dc0c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e0c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc1514f0 0x7f60dc06e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e1600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60dc0c6850 0x7f60dc0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60dbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60dc06e6f0 0x7f60dc0c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38874/0 (socket says 172.21.15.47:38874) 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60dbe00640 1 -- 172.21.15.47:0/968149727 learned_addr learned my addr 172.21.15.47:0/968149727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e1600640 1 --2- 172.21.15.47:0/968149727 >> v2:172.21.15.113:3300/0 conn(0x7f60dc0c6850 0x7f60dc0707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:51782/0 (socket says 172.21.15.47:51782) 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e0c00640 1 -- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc1514f0 msgr2=0x7f60dc06e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:41:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e0c00640 1 -- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc1514f0 msgr2=0x7f60dc06e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e0c00640 1 --2- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc1514f0 0x7f60dc06e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e0c00640 1 --2- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc1514f0 0x7f60dc06e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e1600640 1 -- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc1514f0 msgr2=0x7f60dc06e1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:41:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e1600640 1 --2- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc1514f0 0x7f60dc06e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e1600640 1 -- 172.21.15.47:0/968149727 >> v2:172.21.15.47:3300/0 conn(0x7f60dc06e6f0 msgr2=0x7f60dc0c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e1600640 1 --2- 172.21.15.47:0/968149727 >> v2:172.21.15.47:3300/0 conn(0x7f60dc06e6f0 0x7f60dc0c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60e1600640 1 -- 172.21.15.47:0/968149727 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc15a8a0 con 0x7f60dc0c6850 2024-04-24T22:41:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.719+0000 7f60dbe00640 1 --2- 172.21.15.47:0/968149727 >> v2:172.21.15.47:3300/0 conn(0x7f60dc06e6f0 0x7f60dc0c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.720+0000 7f60e1600640 1 --2- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc0c6850 0x7f60dc0707e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f60dc06d870 tx=0x7f60d402e9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=948323c089363b06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.721+0000 7f60db400640 1 -- 172.21.15.47:0/968149727 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4037020 con 0x7f60dc0c6850 2024-04-24T22:41:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.721+0000 7f60db400640 1 -- 172.21.15.47:0/968149727 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d400d4e0 con 0x7f60dc0c6850 2024-04-24T22:41:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.721+0000 7f60db400640 1 -- 172.21.15.47:0/968149727 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d400b960 con 0x7f60dc0c6850 2024-04-24T22:41:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.721+0000 7f60e2000640 1 -- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc0c6850 msgr2=0x7f60dc0707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.721+0000 7f60e2000640 1 --2- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc0c6850 0x7f60dc0707e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f60dc06d870 tx=0x7f60d402e9e0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.721+0000 7f60e2000640 1 -- 172.21.15.47:0/968149727 shutdown_connections 2024-04-24T22:41:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.721+0000 7f60e2000640 1 --2- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc0c6850 0x7f60dc0707e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.721+0000 7f60e2000640 1 --2- 172.21.15.47:0/968149727 >> v2:172.21.15.47:3300/0 conn(0x7f60dc06e6f0 0x7f60dc0c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.721+0000 7f60e2000640 1 --2- 172.21.15.47:0/968149727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc1514f0 0x7f60dc06e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.721+0000 7f60e2000640 1 -- 172.21.15.47:0/968149727 >> 172.21.15.47:0/968149727 conn(0x7f60dc0cb900 msgr2=0x7f60dc0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.722+0000 7f60e2000640 1 -- 172.21.15.47:0/968149727 shutdown_connections 2024-04-24T22:41:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.722+0000 7f60e2000640 1 -- 172.21.15.47:0/968149727 wait complete. 2024-04-24T22:41:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.723+0000 7f60e2000640 1 Processor -- start 2024-04-24T22:41:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.724+0000 7f60e2000640 1 -- start start 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.724+0000 7f60e2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60dc06e6f0 0x7f60dc14a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.724+0000 7f60e2000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc0c6850 0x7f60dc14bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc1514f0 0x7f60dc144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e2000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc15ae20 con 0x7f60dc06e6f0 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e2000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc15ab40 con 0x7f60dc1514f0 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e2000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc15b100 con 0x7f60dc0c6850 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60dc06e6f0 0x7f60dc14a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60dc06e6f0 0x7f60dc14a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38884/0 (socket says 172.21.15.47:38884) 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e0c00640 1 -- 172.21.15.47:0/147722461 learned_addr learned my addr 172.21.15.47:0/147722461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e1600640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc1514f0 0x7f60dc144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60dbe00640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc0c6850 0x7f60dc14bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60dbe00640 1 -- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc0c6850 msgr2=0x7f60dc14bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60dbe00640 1 -- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc0c6850 msgr2=0x7f60dc14bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60dbe00640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc0c6850 0x7f60dc14bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60dbe00640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc0c6850 0x7f60dc14bd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e0c00640 1 -- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc0c6850 msgr2=0x7f60dc14bd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e0c00640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc0c6850 0x7f60dc14bd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e0c00640 1 -- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc1514f0 msgr2=0x7f60dc144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e0c00640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc1514f0 0x7f60dc144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.725+0000 7f60e0c00640 1 -- 172.21.15.47:0/147722461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc14c3e0 con 0x7f60dc06e6f0 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.726+0000 7f60e0c00640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60dc06e6f0 0x7f60dc14a440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f60d000d7d0 tx=0x7f60d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.726+0000 7f60d8c00640 1 -- 172.21.15.47:0/147722461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d000b6c0 con 0x7f60dc06e6f0 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.726+0000 7f60d8c00640 1 -- 172.21.15.47:0/147722461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d0019070 con 0x7f60dc06e6f0 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.726+0000 7f60d8c00640 1 -- 172.21.15.47:0/147722461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d000bac0 con 0x7f60dc06e6f0 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.726+0000 7f60e2000640 1 -- 172.21.15.47:0/147722461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc14aa20 con 0x7f60dc06e6f0 2024-04-24T22:41:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.726+0000 7f60e2000640 1 -- 172.21.15.47:0/147722461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc14afe0 con 0x7f60dc06e6f0 2024-04-24T22:41:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.727+0000 7f60e2000640 1 -- 172.21.15.47:0/147722461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60dc068d50 con 0x7f60dc06e6f0 2024-04-24T22:41:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.728+0000 7f60d8c00640 1 -- 172.21.15.47:0/147722461 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60d000bc60 con 0x7f60dc06e6f0 2024-04-24T22:41:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.729+0000 7f60d8c00640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f60bc06be30 0x7f60bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.729+0000 7f60d8c00640 1 -- 172.21.15.47:0/147722461 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f60d008b3f0 con 0x7f60dc06e6f0 2024-04-24T22:41:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.730+0000 7f60dbe00640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f60bc06be30 0x7f60bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.730+0000 7f60dbe00640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f60bc06be30 0x7f60bc06e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f60dc14b980 tx=0x7f60c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:57.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:57.734+0000 7f60d8c00640 1 -- 172.21.15.47:0/147722461 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60d0059c80 con 0x7f60dc06e6f0 2024-04-24T22:41:58.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.018+0000 7f60e2000640 1 -- 172.21.15.47:0/147722461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f60dc0c9060 con 0x7f60dc06e6f0 2024-04-24T22:41:58.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.019+0000 7f60d8c00640 1 -- 172.21.15.47:0/147722461 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f60d0059580 con 0x7f60dc06e6f0 2024-04-24T22:41:58.020 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:41:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.021+0000 7f60e2000640 1 -- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f60bc06be30 msgr2=0x7f60bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.021+0000 7f60e2000640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f60bc06be30 0x7f60bc06e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f60dc14b980 tx=0x7f60c8008040 comp rx=0 tx=0).stop 2024-04-24T22:41:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.021+0000 7f60e2000640 1 -- 172.21.15.47:0/147722461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60dc06e6f0 msgr2=0x7f60dc14a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.021+0000 7f60e2000640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60dc06e6f0 0x7f60dc14a440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f60d000d7d0 tx=0x7f60d000dc90 comp rx=0 tx=0).stop 2024-04-24T22:41:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.021+0000 7f60e2000640 1 -- 172.21.15.47:0/147722461 shutdown_connections 2024-04-24T22:41:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.021+0000 7f60e2000640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f60bc06be30 0x7f60bc06e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.021+0000 7f60e2000640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60dc1514f0 0x7f60dc144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.021+0000 7f60e2000640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60dc0c6850 0x7f60dc14bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.021+0000 7f60e2000640 1 --2- 172.21.15.47:0/147722461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60dc06e6f0 0x7f60dc14a440 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.021+0000 7f60e2000640 1 -- 172.21.15.47:0/147722461 >> 172.21.15.47:0/147722461 conn(0x7f60dc0cb900 msgr2=0x7f60dc0707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.022+0000 7f60e2000640 1 -- 172.21.15.47:0/147722461 shutdown_connections 2024-04-24T22:41:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.022+0000 7f60e2000640 1 -- 172.21.15.47:0/147722461 wait complete. 2024-04-24T22:41:58.044 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-24T22:41:58.044 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-24T22:41:58.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:41:58.225 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:41:58.225 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 readers = 3618273.25 kB/sec 2024-04-24T22:41:58.225 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 readers = 3609394.51 kB/sec 2024-04-24T22:41:58.225 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 3618273.25 kB/sec 2024-04-24T22:41:58.226 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 3618273.25 kB/sec 2024-04-24T22:41:58.226 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 3618273.25 kB/sec 2024-04-24T22:41:58.226 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 1048576.00 kB 2024-04-24T22:41:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.244+0000 7f5d83000640 1 Processor -- start 2024-04-24T22:41:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.244+0000 7f5d83000640 1 -- start start 2024-04-24T22:41:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.246+0000 7f5d83000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 0x7f5d7c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.246+0000 7f5d83000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d7c15b2c0 con 0x7f5d7c154560 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.246+0000 7f5d83000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d7c15da90 con 0x7f5d7c073280 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.246+0000 7f5d83000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5d7c15dd70 con 0x7f5d7c0cbb40 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.246+0000 7f5d81c00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5d7c154560 0x7f5d7c0cb430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35418/0 (socket says 172.21.15.47:35418) 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.246+0000 7f5d81c00640 1 -- 172.21.15.47:0/2208390375 learned_addr learned my addr 172.21.15.47:0/2208390375 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d81200640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 0x7f5d7c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d81200640 1 -- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 msgr2=0x7f5d7c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d81200640 1 -- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 msgr2=0x7f5d7c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d81200640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 0x7f5d7c0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d81200640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 0x7f5d7c0cc660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1353359030 0 0) 0x7f5d7c15b2c0 con 0x7f5d7c154560 2024-04-24T22:41:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d64003520 con 0x7f5d7c154560 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2406351951 0 0) 0x7f5d7c15da90 con 0x7f5d7c073280 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d7c15b2c0 con 0x7f5d7c073280 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3070537455 0 0) 0x7f5d64003520 con 0x7f5d7c154560 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 msgr2=0x7f5d7c0cc660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 0x7f5d7c0cc660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 >> v1:172.21.15.113:6789/0 conn(0x7f5d7c073280 legacy=0x7f5d7c0ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7c15e050 con 0x7f5d7c154560 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 567141988 0 0) 0x7f5d70001670 con 0x7f5d7c154560 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 >> v1:172.21.15.47:6789/0 conn(0x7f5d7c154560 legacy=0x7f5d7c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 0x7f5d64003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d640040d0 0x7f5d64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d64004d40 0x7f5d640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c15dd70 con 0x7f5d64004d40 2024-04-24T22:41:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6400bb50 con 0x7f5d640040d0 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.247+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6400bd50 con 0x7f5d7c0cbb40 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.248+0000 7f5d81200640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d640040d0 0x7f5d64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.248+0000 7f5d81c00640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 0x7f5d64003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.248+0000 7f5d82600640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d64004d40 0x7f5d640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.248+0000 7f5d81200640 1 -- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 msgr2=0x7f5d64003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.248+0000 7f5d81200640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 0x7f5d64003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.248+0000 7f5d81200640 1 -- 172.21.15.47:0/2208390375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d64004d40 msgr2=0x7f5d640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.248+0000 7f5d81200640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d64004d40 0x7f5d640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.248+0000 7f5d81200640 1 -- 172.21.15.47:0/2208390375 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d64003520 con 0x7f5d640040d0 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.248+0000 7f5d82600640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d64004d40 0x7f5d640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:58.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.248+0000 7f5d81c00640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c0cbb40 0x7f5d64003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.249+0000 7f5d81200640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d640040d0 0x7f5d64004800 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800b9e0 tx=0x7f5d64005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8c3090e160688f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.249+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68030070 con 0x7f5d640040d0 2024-04-24T22:41:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.249+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d6802ada0 con 0x7f5d640040d0 2024-04-24T22:41:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.249+0000 7f5d7be00640 1 -- 172.21.15.47:0/2208390375 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6800b5e0 con 0x7f5d640040d0 2024-04-24T22:41:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.250+0000 7f5d83000640 1 -- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d640040d0 msgr2=0x7f5d64004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.250+0000 7f5d83000640 1 --2- 172.21.15.47:0/2208390375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d640040d0 0x7f5d64004800 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800b9e0 tx=0x7f5d64005830 comp rx=0 tx=0).stop 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.250+0000 7f5d82600640 1 -- 172.21.15.47:0/2208390375 reap_dead start 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.250+0000 7f5d83000640 1 -- 172.21.15.47:0/2208390375 shutdown_connections 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.250+0000 7f5d83000640 1 -- 172.21.15.47:0/2208390375 >> 172.21.15.47:0/2208390375 conn(0x7f5d7c0672c0 msgr2=0x7f5d7c072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.250+0000 7f5d83000640 1 -- 172.21.15.47:0/2208390375 shutdown_connections 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.250+0000 7f5d83000640 1 -- 172.21.15.47:0/2208390375 wait complete. 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.252+0000 7f5d83000640 1 Processor -- start 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.252+0000 7f5d83000640 1 -- start start 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d83000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640040d0 0x7f5d7c1f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d83000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7c1e3700 0x7f5d7c156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d83000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c158d10 0x7f5d7c159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d83000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c0cd2b0 con 0x7f5d640040d0 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d83000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c0caad0 con 0x7f5d7c1e3700 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d83000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5d7c15ae80 con 0x7f5d7c158d10 2024-04-24T22:41:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d81c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640040d0 0x7f5d7c1f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d81c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640040d0 0x7f5d7c1f0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38916/0 (socket says 172.21.15.47:38916) 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d81c00640 1 -- 172.21.15.47:0/1045832605 learned_addr learned my addr 172.21.15.47:0/1045832605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d81200640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7c1e3700 0x7f5d7c156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d82600640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c158d10 0x7f5d7c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d82600640 1 -- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c158d10 msgr2=0x7f5d7c159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d82600640 1 -- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c158d10 msgr2=0x7f5d7c159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d82600640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c158d10 0x7f5d7c159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d81c00640 1 -- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c158d10 msgr2=0x7f5d7c159190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d82600640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c158d10 0x7f5d7c159190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d81c00640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d7c158d10 0x7f5d7c159190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d81c00640 1 -- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7c1e3700 msgr2=0x7f5d7c156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d81c00640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7c1e3700 0x7f5d7c156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.253+0000 7f5d81c00640 1 -- 172.21.15.47:0/1045832605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7c1596d0 con 0x7f5d640040d0 2024-04-24T22:41:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.254+0000 7f5d81c00640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640040d0 0x7f5d7c1f0da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d700072f0 tx=0x7f5d700077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.254+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7001a070 con 0x7f5d640040d0 2024-04-24T22:41:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.254+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7000fa20 con 0x7f5d640040d0 2024-04-24T22:41:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.254+0000 7f5d83000640 1 -- 172.21.15.47:0/1045832605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7c154940 con 0x7f5d640040d0 2024-04-24T22:41:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.254+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000fdd0 con 0x7f5d640040d0 2024-04-24T22:41:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.254+0000 7f5d83000640 1 -- 172.21.15.47:0/1045832605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7c154be0 con 0x7f5d640040d0 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.255+0000 7f5d83000640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c158450 0x7f5d7c14a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.255+0000 7f5d83000640 1 -- 172.21.15.47:0/1045832605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c0c6080 con 0x7f5d7c158450 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.255+0000 7f5d81200640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c158450 0x7f5d7c14a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.256+0000 7f5d81200640 1 -- 172.21.15.47:0/1045832605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d7c04f5e0 con 0x7f5d7c158450 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.256+0000 7f5d81200640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c158450 0x7f5d7c14a500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d64004780 tx=0x7f5d68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.256+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d7001d070 con 0x7f5d640040d0 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.257+0000 7f5d79600640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5d5c06be60 0x7f5d5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.257+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5d7008a9a0 con 0x7f5d640040d0 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.257+0000 7f5d82600640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5d5c06be60 0x7f5d5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.257+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68034d60 con 0x7f5d7c158450 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.258+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5d68047440 con 0x7f5d7c158450 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.258+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c158450 msgr2=0x7f5d7c14a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.258+0000 7f5d79600640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c158450 0x7f5d7c14a500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d64004780 tx=0x7f5d68004730 comp rx=0 tx=0).stop 2024-04-24T22:41:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.258+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c158450 msgr2=0x7f5d7c14a500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.258+0000 7f5d79600640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c158450 0x7f5d7c14a500 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5d64004780 tx=0x7f5d68004730 comp rx=0 tx=0).stop 2024-04-24T22:41:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.258+0000 7f5d82600640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5d5c06be60 0x7f5d5c06e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c06c300 tx=0x7f5d74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.265+0000 7f5d83000640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c14aa40 0x7f5d7c1f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.265+0000 7f5d83000640 1 -- 172.21.15.47:0/1045832605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c0ca080 con 0x7f5d7c14aa40 2024-04-24T22:41:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.265+0000 7f5d81200640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c14aa40 0x7f5d7c1f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.265+0000 7f5d81200640 1 -- 172.21.15.47:0/1045832605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5d7c153d60 con 0x7f5d7c14aa40 2024-04-24T22:41:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.266+0000 7f5d81200640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c14aa40 0x7f5d7c1f38e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006b10 tx=0x7f5d68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.266+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68052430 con 0x7f5d7c14aa40 2024-04-24T22:41:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.266+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5039 (secure 0 0 0) 0x7f5d6803d3f0 con 0x7f5d7c14aa40 2024-04-24T22:41:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.266+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c14aa40 msgr2=0x7f5d7c1f38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.266+0000 7f5d79600640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c14aa40 0x7f5d7c1f38e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006b10 tx=0x7f5d68008040 comp rx=0 tx=0).stop 2024-04-24T22:41:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.266+0000 7f5d79600640 1 -- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c14aa40 msgr2=0x7f5d7c1f38e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.266+0000 7f5d79600640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d7c14aa40 0x7f5d7c1f38e0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006b10 tx=0x7f5d68008040 comp rx=0 tx=0).stop 2024-04-24T22:41:58.268 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:58.268 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:58.268 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 20, 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 58303, 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:58.269 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:58.270 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:58.271 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:58.272 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.273 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:58.274 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:58.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.276 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:58.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:58.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.279 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:58.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.281 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.281 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.281 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-24T22:41:58.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.281 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.281 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:58.281 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.267+0000 7f5d83000640 1 -- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5d5c06be60 msgr2=0x7f5d5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.267+0000 7f5d83000640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f5d5c06be60 0x7f5d5c06e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c06c300 tx=0x7f5d74009040 comp rx=0 tx=0).stop 2024-04-24T22:41:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.267+0000 7f5d83000640 1 -- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640040d0 msgr2=0x7f5d7c1f0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.267+0000 7f5d83000640 1 --2- 172.21.15.47:0/1045832605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d640040d0 0x7f5d7c1f0da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d700072f0 tx=0x7f5d700077b0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.267+0000 7f5d82600640 1 -- 172.21.15.47:0/1045832605 reap_dead start 2024-04-24T22:41:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.267+0000 7f5d82600640 1 -- 172.21.15.47:0/1045832605 reap_dead start 2024-04-24T22:41:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.267+0000 7f5d83000640 1 -- 172.21.15.47:0/1045832605 shutdown_connections 2024-04-24T22:41:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.267+0000 7f5d83000640 1 -- 172.21.15.47:0/1045832605 >> 172.21.15.47:0/1045832605 conn(0x7f5d7c0672c0 msgr2=0x7f5d7c072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.267+0000 7f5d83000640 1 -- 172.21.15.47:0/1045832605 shutdown_connections 2024-04-24T22:41:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.267+0000 7f5d83000640 1 -- 172.21.15.47:0/1045832605 wait complete. 2024-04-24T22:41:58.283 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:41:58.284 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-24T22:41:58.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:41:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.479+0000 7fb98ca00640 1 Processor -- start 2024-04-24T22:41:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.479+0000 7fb98ca00640 1 -- start start 2024-04-24T22:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.480+0000 7fb98ca00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb988152960 0x7fb9880c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.480+0000 7fb98ca00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9880c9400 con 0x7fb988072b80 2024-04-24T22:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.480+0000 7fb98ca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9880c7e50 con 0x7fb9880c8fa0 2024-04-24T22:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.480+0000 7fb98ca00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb988158f20 con 0x7fb988152960 2024-04-24T22:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.481+0000 7fb985e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb988152960 0x7fb9880c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.481+0000 7fb985e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb988152960 0x7fb9880c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38954/0 (socket says 172.21.15.47:38954) 2024-04-24T22:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.481+0000 7fb986800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb988072b80 0x7fb988162db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35426/0 (socket says 172.21.15.47:35426) 2024-04-24T22:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.481+0000 7fb985e00640 1 -- 172.21.15.47:0/1286282373 learned_addr learned my addr 172.21.15.47:0/1286282373 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.481+0000 7fb985e00640 1 -- 172.21.15.47:0/1286282373 >> v1:172.21.15.113:6789/0 conn(0x7fb9880c8fa0 legacy=0x7fb98815ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.481+0000 7fb985e00640 1 -- 172.21.15.47:0/1286282373 >> v1:172.21.15.47:6789/0 conn(0x7fb988072b80 legacy=0x7fb988162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.481+0000 7fb985e00640 1 -- 172.21.15.47:0/1286282373 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9881585f0 con 0x7fb988152960 2024-04-24T22:41:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.481+0000 7fb984a00640 1 -- 172.21.15.47:0/1286282373 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081265834 0 0) 0x7fb9880c7e50 con 0x7fb9880c8fa0 2024-04-24T22:41:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.482+0000 7fb985e00640 1 --2- 172.21.15.47:0/1286282373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988152960 0x7fb9880c7750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb978009e50 tx=0x7fb97802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d679f1e26621123 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.483+0000 7fb984a00640 1 -- 172.21.15.47:0/1286282373 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978036070 con 0x7fb988152960 2024-04-24T22:41:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.483+0000 7fb984a00640 1 -- 172.21.15.47:0/1286282373 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb978031600 con 0x7fb988152960 2024-04-24T22:41:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.483+0000 7fb984a00640 1 -- 172.21.15.47:0/1286282373 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780319d0 con 0x7fb988152960 2024-04-24T22:41:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.483+0000 7fb98ca00640 1 -- 172.21.15.47:0/1286282373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988152960 msgr2=0x7fb9880c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.483+0000 7fb98ca00640 1 --2- 172.21.15.47:0/1286282373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988152960 0x7fb9880c7750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb978009e50 tx=0x7fb97802a750 comp rx=0 tx=0).stop 2024-04-24T22:41:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.483+0000 7fb98ca00640 1 -- 172.21.15.47:0/1286282373 shutdown_connections 2024-04-24T22:41:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.483+0000 7fb98ca00640 1 --2- 172.21.15.47:0/1286282373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988152960 0x7fb9880c7750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.483+0000 7fb98ca00640 1 -- 172.21.15.47:0/1286282373 >> 172.21.15.47:0/1286282373 conn(0x7fb98806e580 msgr2=0x7fb98806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.483+0000 7fb98ca00640 1 -- 172.21.15.47:0/1286282373 shutdown_connections 2024-04-24T22:41:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.484+0000 7fb98ca00640 1 -- 172.21.15.47:0/1286282373 wait complete. 2024-04-24T22:41:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.485+0000 7fb98ca00640 1 Processor -- start 2024-04-24T22:41:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.485+0000 7fb98ca00640 1 -- start start 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb98ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988072b80 0x7fb98815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb98ca00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9880c8fa0 0x7fb9881ee770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb98ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988152960 0x7fb988147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb98ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb9880c8130 con 0x7fb988072b80 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb98ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb9881593b0 con 0x7fb988152960 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb98ca00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb988158bb0 con 0x7fb9880c8fa0 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb985e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988072b80 0x7fb98815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb986800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988152960 0x7fb988147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb985e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988072b80 0x7fb98815c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38958/0 (socket says 172.21.15.47:38958) 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb986800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988152960 0x7fb988147620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:51828/0 (socket says 172.21.15.47:51828) 2024-04-24T22:41:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb985e00640 1 -- 172.21.15.47:0/1326373835 learned_addr learned my addr 172.21.15.47:0/1326373835 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.486+0000 7fb985400640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9880c8fa0 0x7fb9881ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.487+0000 7fb985e00640 1 -- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9880c8fa0 msgr2=0x7fb9881ee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.487+0000 7fb985e00640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9880c8fa0 0x7fb9881ee770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.487+0000 7fb985e00640 1 -- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988152960 msgr2=0x7fb988147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.487+0000 7fb985e00640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988152960 0x7fb988147620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.487+0000 7fb985e00640 1 -- 172.21.15.47:0/1326373835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9880c73e0 con 0x7fb988072b80 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.487+0000 7fb986800640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988152960 0x7fb988147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.487+0000 7fb985e00640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988072b80 0x7fb98815c290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb978034040 tx=0x7fb978004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.487+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978041030 con 0x7fb988072b80 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.487+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb978031400 con 0x7fb988072b80 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.487+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978036040 con 0x7fb988072b80 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.488+0000 7fb98ca00640 1 -- 172.21.15.47:0/1326373835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9881eecb0 con 0x7fb988072b80 2024-04-24T22:41:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.488+0000 7fb98ca00640 1 -- 172.21.15.47:0/1326373835 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9880c36b0 con 0x7fb988072b80 2024-04-24T22:41:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.489+0000 7fb98ca00640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9880c4020 0x7fb988147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.489+0000 7fb98ca00640 1 -- 172.21.15.47:0/1326373835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb988072980 con 0x7fb9880c4020 2024-04-24T22:41:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.489+0000 7fb985400640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9880c4020 0x7fb988147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.489+0000 7fb985400640 1 -- 172.21.15.47:0/1326373835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9880c3880 con 0x7fb9880c4020 2024-04-24T22:41:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.489+0000 7fb985400640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9880c4020 0x7fb988147b60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb98806bb30 tx=0x7fb97000fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.490+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb978044070 con 0x7fb988072b80 2024-04-24T22:41:58.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.491+0000 7fb97e000640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb96406be30 0x7fb96406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.491+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9700050d0 con 0x7fb9880c4020 2024-04-24T22:41:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.491+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb9780abe90 con 0x7fb988072b80 2024-04-24T22:41:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.491+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb970018070 con 0x7fb9880c4020 2024-04-24T22:41:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.491+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9880c4020 msgr2=0x7fb988147b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.491+0000 7fb97e000640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9880c4020 0x7fb988147b60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb98806bb30 tx=0x7fb97000fc00 comp rx=0 tx=0).stop 2024-04-24T22:41:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.491+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9880c4020 msgr2=0x7fb988147b60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.491+0000 7fb97e000640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9880c4020 0x7fb988147b60 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb98806bb30 tx=0x7fb97000fc00 comp rx=0 tx=0).stop 2024-04-24T22:41:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.491+0000 7fb986800640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb96406be30 0x7fb96406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.492+0000 7fb986800640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb96406be30 0x7fb96406e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb980000f60 tx=0x7fb980007da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.498+0000 7fb98ca00640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988148140 0x7fb9881edd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.498+0000 7fb98ca00640 1 -- 172.21.15.47:0/1326373835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb988073b90 con 0x7fb988148140 2024-04-24T22:41:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.498+0000 7fb985400640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988148140 0x7fb9881edd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.499+0000 7fb985400640 1 -- 172.21.15.47:0/1326373835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb98806c200 con 0x7fb988148140 2024-04-24T22:41:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.499+0000 7fb985400640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988148140 0x7fb9881edd60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb970014000 tx=0x7fb970001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.499+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97001ee10 con 0x7fb988148140 2024-04-24T22:41:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5065 (secure 0 0 0) 0x7fb97001ee10 con 0x7fb988148140 2024-04-24T22:41:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988148140 msgr2=0x7fb9881edd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb97e000640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988148140 0x7fb9881edd60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb970014000 tx=0x7fb970001080 comp rx=0 tx=0).stop 2024-04-24T22:41:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb97e000640 1 -- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988148140 msgr2=0x7fb9881edd60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb97e000640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb988148140 0x7fb9881edd60 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb970014000 tx=0x7fb970001080 comp rx=0 tx=0).stop 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 20, 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 160067, 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:58.502 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:58.503 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:58.504 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:58.505 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.506 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.507 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:58.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:58.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.511 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7 2024-04-24T22:41:58.512 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.513 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.514 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:58.514 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb98ca00640 1 -- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb96406be30 msgr2=0x7fb96406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb98ca00640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb96406be30 0x7fb96406e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb980000f60 tx=0x7fb980007da0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb98ca00640 1 -- 172.21.15.47:0/1326373835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988072b80 msgr2=0x7fb98815c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb98ca00640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988072b80 0x7fb98815c290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb978034040 tx=0x7fb978004a40 comp rx=0 tx=0).stop 2024-04-24T22:41:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb986800640 1 -- 172.21.15.47:0/1326373835 reap_dead start 2024-04-24T22:41:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb98ca00640 1 -- 172.21.15.47:0/1326373835 shutdown_connections 2024-04-24T22:41:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb98ca00640 1 --2- 172.21.15.47:0/1326373835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb988072b80 0x7fb98815c290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb98ca00640 1 -- 172.21.15.47:0/1326373835 >> 172.21.15.47:0/1326373835 conn(0x7fb98806e580 msgr2=0x7fb98814ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb98ca00640 1 -- 172.21.15.47:0/1326373835 shutdown_connections 2024-04-24T22:41:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.500+0000 7fb98ca00640 1 -- 172.21.15.47:0/1326373835 wait complete. 2024-04-24T22:41:58.517 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:41:58.518 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-24T22:41:58.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-24T22:41:58.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.714+0000 7fe814000640 1 Processor -- start 2024-04-24T22:41:58.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.714+0000 7fe814000640 1 -- start start 2024-04-24T22:41:58.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.716+0000 7fe814000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe80c0c1a00 0x7fe80c072f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.716+0000 7fe814000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80c0738f0 con 0x7fe80c151780 2024-04-24T22:41:58.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.716+0000 7fe814000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80c15aa30 con 0x7fe80c0734a0 2024-04-24T22:41:58.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.716+0000 7fe814000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe80c15ad10 con 0x7fe80c0c1a00 2024-04-24T22:41:58.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.716+0000 7fe811600640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe80c151780 0x7fe80c0c8590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35440/0 (socket says 172.21.15.47:35440) 2024-04-24T22:41:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.716+0000 7fe811600640 1 -- 172.21.15.47:0/2128790607 learned_addr learned my addr 172.21.15.47:0/2128790607 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.716+0000 7fe810c00640 1 --2- 172.21.15.47:0/2128790607 >> v2:172.21.15.113:3300/0 conn(0x7fe80c0c1a00 0x7fe80c072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.717+0000 7fe80be00640 1 -- 172.21.15.47:0/2128790607 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 687674422 0 0) 0x7fe800001110 con 0x7fe80c151780 2024-04-24T22:41:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.717+0000 7fe80be00640 1 -- 172.21.15.47:0/2128790607 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7f4005730 con 0x7fe80c151780 2024-04-24T22:41:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.717+0000 7fe810c00640 1 -- 172.21.15.47:0/2128790607 >> v1:172.21.15.113:6789/0 conn(0x7fe80c0734a0 legacy=0x7fe80c1576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.717+0000 7fe80be00640 1 -- 172.21.15.47:0/2128790607 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2124916863 0 0) 0x7fe80c15aa30 con 0x7fe80c0734a0 2024-04-24T22:41:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.717+0000 7fe810c00640 1 -- 172.21.15.47:0/2128790607 >> v1:172.21.15.47:6789/0 conn(0x7fe80c151780 legacy=0x7fe80c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.717+0000 7fe810c00640 1 -- 172.21.15.47:0/2128790607 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c15aff0 con 0x7fe80c0c1a00 2024-04-24T22:41:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.718+0000 7fe810c00640 1 --2- 172.21.15.47:0/2128790607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0c1a00 0x7fe80c072f60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80097a0 tx=0x7fe7f802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28875bf07b2d51e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.718+0000 7fe80be00640 1 -- 172.21.15.47:0/2128790607 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8037070 con 0x7fe80c0c1a00 2024-04-24T22:41:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.718+0000 7fe80be00640 1 -- 172.21.15.47:0/2128790607 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f8032640 con 0x7fe80c0c1a00 2024-04-24T22:41:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.718+0000 7fe80be00640 1 -- 172.21.15.47:0/2128790607 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8032a10 con 0x7fe80c0c1a00 2024-04-24T22:41:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.719+0000 7fe814000640 1 -- 172.21.15.47:0/2128790607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0c1a00 msgr2=0x7fe80c072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.719+0000 7fe814000640 1 --2- 172.21.15.47:0/2128790607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0c1a00 0x7fe80c072f60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80097a0 tx=0x7fe7f802a2c0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.719+0000 7fe814000640 1 -- 172.21.15.47:0/2128790607 shutdown_connections 2024-04-24T22:41:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.719+0000 7fe814000640 1 --2- 172.21.15.47:0/2128790607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0c1a00 0x7fe80c072f60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.719+0000 7fe814000640 1 -- 172.21.15.47:0/2128790607 >> 172.21.15.47:0/2128790607 conn(0x7fe80c06e580 msgr2=0x7fe80c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.719+0000 7fe814000640 1 -- 172.21.15.47:0/2128790607 shutdown_connections 2024-04-24T22:41:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.719+0000 7fe814000640 1 -- 172.21.15.47:0/2128790607 wait complete. 2024-04-24T22:41:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.721+0000 7fe814000640 1 Processor -- start 2024-04-24T22:41:58.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.722+0000 7fe814000640 1 -- start start 2024-04-24T22:41:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe814000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0734a0 0x7fe80c13e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe814000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c0c1a00 0x7fe80c1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe814000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c151780 0x7fe80c144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe814000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c0c8ca0 con 0x7fe80c151780 2024-04-24T22:41:58.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe814000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c15b5b0 con 0x7fe80c0734a0 2024-04-24T22:41:58.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe814000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c0c8e20 con 0x7fe80c0c1a00 2024-04-24T22:41:58.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe811600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0734a0 0x7fe80c13e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe812000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c151780 0x7fe80c144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe811600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0734a0 0x7fe80c13e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:51872/0 (socket says 172.21.15.47:51872) 2024-04-24T22:41:58.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe810c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c0c1a00 0x7fe80c1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.723+0000 7fe811600640 1 -- 172.21.15.47:0/4245112849 learned_addr learned my addr 172.21.15.47:0/4245112849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe811600640 1 -- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c0c1a00 msgr2=0x7fe80c1593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe811600640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c0c1a00 0x7fe80c1593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe811600640 1 -- 172.21.15.47:0/4245112849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c151780 msgr2=0x7fe80c144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe811600640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c151780 0x7fe80c144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe811600640 1 -- 172.21.15.47:0/4245112849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c13e610 con 0x7fe80c0734a0 2024-04-24T22:41:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe812000640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe80c151780 0x7fe80c144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:41:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe811600640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0734a0 0x7fe80c13e0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe80000c700 tx=0x7fe80000cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800005ab0 con 0x7fe80c0734a0 2024-04-24T22:41:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe800005c70 con 0x7fe80c0734a0 2024-04-24T22:41:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8000072b0 con 0x7fe80c0734a0 2024-04-24T22:41:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.724+0000 7fe814000640 1 -- 172.21.15.47:0/4245112849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c159be0 con 0x7fe80c0734a0 2024-04-24T22:41:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.725+0000 7fe814000640 1 -- 172.21.15.47:0/4245112849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c143f50 con 0x7fe80c0734a0 2024-04-24T22:41:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.726+0000 7fe814000640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144190 0x7fe80c13d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.726+0000 7fe814000640 1 -- 172.21.15.47:0/4245112849 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c15a4a0 con 0x7fe80c144190 2024-04-24T22:41:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.726+0000 7fe810c00640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144190 0x7fe80c13d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.726+0000 7fe810c00640 1 -- 172.21.15.47:0/4245112849 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe80c143a20 con 0x7fe80c144190 2024-04-24T22:41:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.727+0000 7fe810c00640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144190 0x7fe80c13d350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe80c073ad0 tx=0x7fe7f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.727+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8042070 con 0x7fe80c144190 2024-04-24T22:41:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.727+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe7f8003da0 con 0x7fe80c144190 2024-04-24T22:41:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.727+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144190 msgr2=0x7fe80c13d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.727+0000 7fe809600640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144190 0x7fe80c13d350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe80c073ad0 tx=0x7fe7f80046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.727+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144190 msgr2=0x7fe80c13d350 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.727+0000 7fe809600640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144190 0x7fe80c13d350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.727+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe800007450 con 0x7fe80c0734a0 2024-04-24T22:41:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.728+0000 7fe809600640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe7ec06b8a0 0x7fe7ec06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.728+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe80008aa60 con 0x7fe80c0734a0 2024-04-24T22:41:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.729+0000 7fe810c00640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe7ec06b8a0 0x7fe7ec06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.729+0000 7fe810c00640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe7ec06b8a0 0x7fe7ec06dd60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8041000 tx=0x7fe7f803e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.735+0000 7fe814000640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144570 0x7fe80c13d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.735+0000 7fe814000640 1 -- 172.21.15.47:0/4245112849 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c073b90 con 0x7fe80c144570 2024-04-24T22:41:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.735+0000 7fe812000640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144570 0x7fe80c13d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.736+0000 7fe812000640 1 -- 172.21.15.47:0/4245112849 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe80c150fa0 con 0x7fe80c144570 2024-04-24T22:41:58.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.736+0000 7fe812000640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144570 0x7fe80c13d890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe80400fc20 tx=0x7fe80400d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:58.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.736+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80400dc20 con 0x7fe80c144570 2024-04-24T22:41:58.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4554 (secure 0 0 0) 0x7fe80400dc20 con 0x7fe80c144570 2024-04-24T22:41:58.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144570 msgr2=0x7fe80c13d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe809600640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144570 0x7fe80c13d890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe80400fc20 tx=0x7fe80400d520 comp rx=0 tx=0).stop 2024-04-24T22:41:58.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe809600640 1 -- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144570 msgr2=0x7fe80c13d890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:41:58.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe809600640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80c144570 0x7fe80c13d890 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe80400fc20 tx=0x7fe80400d520 comp rx=0 tx=0).stop 2024-04-24T22:41:58.852 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:41:58.852 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 20, 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 10385, 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:41:58.853 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:41:58.854 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:41:58.855 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:41:58.856 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:41:58.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.858 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:58.860 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:41:58.861 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.862 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:41:58.863 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:41:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe814000640 1 -- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe7ec06b8a0 msgr2=0x7fe7ec06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe814000640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe7ec06b8a0 0x7fe7ec06dd60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8041000 tx=0x7fe7f803e000 comp rx=0 tx=0).stop 2024-04-24T22:41:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe814000640 1 -- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0734a0 msgr2=0x7fe80c13e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe814000640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0734a0 0x7fe80c13e0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe80000c700 tx=0x7fe80000cbc0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe812000640 1 -- 172.21.15.47:0/4245112849 reap_dead start 2024-04-24T22:41:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe814000640 1 -- 172.21.15.47:0/4245112849 shutdown_connections 2024-04-24T22:41:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe814000640 1 --2- 172.21.15.47:0/4245112849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe80c0734a0 0x7fe80c13e0d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe814000640 1 -- 172.21.15.47:0/4245112849 >> 172.21.15.47:0/4245112849 conn(0x7fe80c06e580 msgr2=0x7fe80c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe814000640 1 -- 172.21.15.47:0/4245112849 shutdown_connections 2024-04-24T22:41:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:58.737+0000 7fe814000640 1 -- 172.21.15.47:0/4245112849 wait complete. 2024-04-24T22:41:58.866 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:41:58.866 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-24T22:41:58.866 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-24T22:41:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.083+0000 7feabb200640 1 Processor -- start 2024-04-24T22:41:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.084+0000 7feabb200640 1 -- start start 2024-04-24T22:41:59.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.085+0000 7feabb200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feabc154560 0x7feabc0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:59.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.085+0000 7feabb200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc0cb970 0x7feabc0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.085+0000 7feabb200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feabc0cbe00 con 0x7feabc073280 2024-04-24T22:41:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.085+0000 7feabb200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7feabc15b2c0 con 0x7feabc154560 2024-04-24T22:41:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.085+0000 7feabb200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7feabc15da90 con 0x7feabc0cb970 2024-04-24T22:41:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.086+0000 7feab9e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feabc154560 0x7feabc0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.086+0000 7feab9400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feabc073280 0x7feabc0cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:35452/0 (socket says 172.21.15.47:35452) 2024-04-24T22:41:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.086+0000 7feab9400640 1 -- 172.21.15.47:0/4020518366 learned_addr learned my addr 172.21.15.47:0/4020518366 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:41:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.086+0000 7feaba800640 1 --2- 172.21.15.47:0/4020518366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc0cb970 0x7feabc0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.086+0000 7feab9e00640 1 -- 172.21.15.47:0/4020518366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc0cb970 msgr2=0x7feabc0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:41:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.086+0000 7feab9e00640 1 --2- 172.21.15.47:0/4020518366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc0cb970 0x7feabc0ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.086+0000 7feab9e00640 1 -- 172.21.15.47:0/4020518366 >> v1:172.21.15.47:6789/0 conn(0x7feabc073280 legacy=0x7feabc0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.086+0000 7feab9e00640 1 -- 172.21.15.47:0/4020518366 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc15dd70 con 0x7feabc154560 2024-04-24T22:41:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.087+0000 7feab9e00640 1 --2- 172.21.15.47:0/4020518366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feabc154560 0x7feabc0cb430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feaac0097a0 tx=0x7feaac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5f60f58e315cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.088+0000 7feab8a00640 1 -- 172.21.15.47:0/4020518366 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac037070 con 0x7feabc154560 2024-04-24T22:41:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.088+0000 7feab8a00640 1 -- 172.21.15.47:0/4020518366 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaac032440 con 0x7feabc154560 2024-04-24T22:41:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.088+0000 7feab8a00640 1 -- 172.21.15.47:0/4020518366 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac032810 con 0x7feabc154560 2024-04-24T22:41:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.088+0000 7feabb200640 1 -- 172.21.15.47:0/4020518366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feabc154560 msgr2=0x7feabc0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.088+0000 7feabb200640 1 --2- 172.21.15.47:0/4020518366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feabc154560 0x7feabc0cb430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feaac0097a0 tx=0x7feaac02a180 comp rx=0 tx=0).stop 2024-04-24T22:41:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.088+0000 7feabb200640 1 -- 172.21.15.47:0/4020518366 shutdown_connections 2024-04-24T22:41:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.088+0000 7feabb200640 1 --2- 172.21.15.47:0/4020518366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc0cb970 0x7feabc0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.088+0000 7feabb200640 1 --2- 172.21.15.47:0/4020518366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feabc154560 0x7feabc0cb430 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.088+0000 7feabb200640 1 -- 172.21.15.47:0/4020518366 >> 172.21.15.47:0/4020518366 conn(0x7feabc0672c0 msgr2=0x7feabc072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.089+0000 7feabb200640 1 -- 172.21.15.47:0/4020518366 shutdown_connections 2024-04-24T22:41:59.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.089+0000 7feabb200640 1 -- 172.21.15.47:0/4020518366 wait complete. 2024-04-24T22:41:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.090+0000 7feabb200640 1 Processor -- start 2024-04-24T22:41:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.090+0000 7feabb200640 1 -- start start 2024-04-24T22:41:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.091+0000 7feabb200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc073280 0x7feabc0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.091+0000 7feabb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feabc0cb970 0x7feabc15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.091+0000 7feabb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feabc154560 0x7feabc15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.091+0000 7feabb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feabc0caad0 con 0x7feabc0cb970 2024-04-24T22:41:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.091+0000 7feabb200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feabc15e330 con 0x7feabc154560 2024-04-24T22:41:59.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.091+0000 7feabb200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7feabc0cd2b0 con 0x7feabc073280 2024-04-24T22:41:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.092+0000 7feab9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feabc0cb970 0x7feabc15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.092+0000 7feab9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc073280 0x7feabc0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.092+0000 7feaba800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feabc154560 0x7feabc15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.092+0000 7feab9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc073280 0x7feabc0cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:41288/0 (socket says 172.21.15.47:41288) 2024-04-24T22:41:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.092+0000 7feab9e00640 1 -- 172.21.15.47:0/1832100428 learned_addr learned my addr 172.21.15.47:0/1832100428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:41:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.092+0000 7feab9e00640 1 -- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feabc154560 msgr2=0x7feabc15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.092+0000 7feab9e00640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feabc154560 0x7feabc15cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.092+0000 7feab9e00640 1 -- 172.21.15.47:0/1832100428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feabc0cb970 msgr2=0x7feabc15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.092+0000 7feab9e00640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feabc0cb970 0x7feabc15c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.092+0000 7feab9e00640 1 -- 172.21.15.47:0/1832100428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc1596a0 con 0x7feabc073280 2024-04-24T22:41:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.093+0000 7feab9e00640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc073280 0x7feabc0cdd00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feaac0098c0 tx=0x7feaac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:59.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.093+0000 7feab2000640 1 -- 172.21.15.47:0/1832100428 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac037040 con 0x7feabc073280 2024-04-24T22:41:59.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.093+0000 7feab2000640 1 -- 172.21.15.47:0/1832100428 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaac030500 con 0x7feabc073280 2024-04-24T22:41:59.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.093+0000 7feab2000640 1 -- 172.21.15.47:0/1832100428 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac0308d0 con 0x7feabc073280 2024-04-24T22:41:59.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.093+0000 7feabb200640 1 -- 172.21.15.47:0/1832100428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc159980 con 0x7feabc073280 2024-04-24T22:41:59.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.093+0000 7feabb200640 1 -- 172.21.15.47:0/1832100428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc159f40 con 0x7feabc073280 2024-04-24T22:41:59.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.094+0000 7feabb200640 1 -- 172.21.15.47:0/1832100428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feabc0c6080 con 0x7feabc073280 2024-04-24T22:41:59.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.095+0000 7feab2000640 1 -- 172.21.15.47:0/1832100428 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feaac044020 con 0x7feabc073280 2024-04-24T22:41:59.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.096+0000 7feab2000640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea9c06be60 0x7fea9c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:41:59.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.096+0000 7feab2000640 1 -- 172.21.15.47:0/1832100428 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7feaac0aaff0 con 0x7feabc073280 2024-04-24T22:41:59.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.096+0000 7feab9400640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea9c06be60 0x7fea9c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:41:59.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.097+0000 7feab9400640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea9c06be60 0x7fea9c06e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feabc15b790 tx=0x7feaa8001990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:41:59.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.101+0000 7feab2000640 1 -- 172.21.15.47:0/1832100428 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feaac0792c0 con 0x7feabc073280 2024-04-24T22:41:59.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.380+0000 7feabb200640 1 -- 172.21.15.47:0/1832100428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7feabc04f5e0 con 0x7feabc073280 2024-04-24T22:41:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.383+0000 7feab2000640 1 -- 172.21.15.47:0/1832100428 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7feaac07d5c0 con 0x7feabc073280 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.386+0000 7feabb200640 1 -- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea9c06be60 msgr2=0x7fea9c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea9c06be60 0x7fea9c06e320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feabc15b790 tx=0x7feaa8001990 comp rx=0 tx=0).stop 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 -- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc073280 msgr2=0x7feabc0cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc073280 0x7feabc0cdd00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feaac0098c0 tx=0x7feaac0046c0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 -- 172.21.15.47:0/1832100428 shutdown_connections 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea9c06be60 0x7fea9c06e320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feabc154560 0x7feabc15cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feabc0cb970 0x7feabc15c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 --2- 172.21.15.47:0/1832100428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feabc073280 0x7feabc0cdd00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 -- 172.21.15.47:0/1832100428 >> 172.21.15.47:0/1832100428 conn(0x7feabc0672c0 msgr2=0x7feabc0729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 -- 172.21.15.47:0/1832100428 shutdown_connections 2024-04-24T22:41:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:41:59.387+0000 7feabb200640 1 -- 172.21.15.47:0/1832100428 wait complete. 2024-04-24T22:41:59.409 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-24T22:42:00.700 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:00.700 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 re-readers = 2638052.25 kB/sec 2024-04-24T22:42:00.700 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 re-readers = 2633039.21 kB/sec 2024-04-24T22:42:00.700 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 2638052.25 kB/sec 2024-04-24T22:42:00.700 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 2638052.25 kB/sec 2024-04-24T22:42:00.700 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 2638052.25 kB/sec 2024-04-24T22:42:00.700 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 1048576.00 kB 2024-04-24T22:42:07.224 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:07.224 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:07.224 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:07.224 INFO:tasks.workunit.client.0.smithi047.stdout:iozone test complete. 2024-04-24T22:42:07.230 INFO:tasks.workunit.client.0.smithi047.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-24T22:42:07.321 INFO:tasks.workunit.client.0.smithi047.stdout: Iozone: Performance Test of File I/O 2024-04-24T22:42:07.321 INFO:tasks.workunit.client.0.smithi047.stdout: Version $Revision: 3.494 $ 2024-04-24T22:42:07.322 INFO:tasks.workunit.client.0.smithi047.stdout: Compiled for 64 bit mode. 2024-04-24T22:42:07.322 INFO:tasks.workunit.client.0.smithi047.stdout: Build: linux-AMD64 2024-04-24T22:42:07.322 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:07.322 INFO:tasks.workunit.client.0.smithi047.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-24T22:42:07.322 INFO:tasks.workunit.client.0.smithi047.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-24T22:42:07.322 INFO:tasks.workunit.client.0.smithi047.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-24T22:42:07.322 INFO:tasks.workunit.client.0.smithi047.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-24T22:42:07.322 INFO:tasks.workunit.client.0.smithi047.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-24T22:42:07.322 INFO:tasks.workunit.client.0.smithi047.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Run began: Wed Apr 24 22:42:07 2024 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Include close in write timing 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Include fsync in write timing 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: File size set to 1048576 kB 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Record Size 1024 kB 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Output is in kBytes/sec 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Time Resolution = 0.000001 seconds. 2024-04-24T22:42:07.323 INFO:tasks.workunit.client.0.smithi047.stdout: Processor cache size set to 1024 kBytes. 2024-04-24T22:42:07.324 INFO:tasks.workunit.client.0.smithi047.stdout: Processor cache line size set to 32 bytes. 2024-04-24T22:42:07.324 INFO:tasks.workunit.client.0.smithi047.stdout: File stride size set to 17 * record size. 2024-04-24T22:42:07.324 INFO:tasks.workunit.client.0.smithi047.stdout: Throughput test with 1 process 2024-04-24T22:42:07.324 INFO:tasks.workunit.client.0.smithi047.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-04-24T22:42:09.409 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-24T22:42:09.410 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:42:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.690+0000 7ff512400640 1 Processor -- start 2024-04-24T22:42:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.690+0000 7ff512400640 1 -- start start 2024-04-24T22:42:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.692+0000 7ff512400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff50c143a60 0x7ff50c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.692+0000 7ff512400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff50c0c2e80 0x7ff50c0c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.692+0000 7ff512400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff50c0c7fc0 con 0x7ff50c0c9c80 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.692+0000 7ff512400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff50c15b3e0 con 0x7ff50c143a60 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.692+0000 7ff512400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c15ab40 con 0x7ff50c0c2e80 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.692+0000 7ff50b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff50c143a60 0x7ff50c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.692+0000 7ff50b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff50c143a60 0x7ff50c15a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60354/0 (socket says 172.21.15.47:60354) 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.692+0000 7ff50aa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff50c0c2e80 0x7ff50c0c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.692+0000 7ff50b400640 1 -- 172.21.15.47:0/774843411 learned_addr learned my addr 172.21.15.47:0/774843411 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.693+0000 7ff50b400640 1 -- 172.21.15.47:0/774843411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff50c0c2e80 msgr2=0x7ff50c0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.693+0000 7ff50b400640 1 --2- 172.21.15.47:0/774843411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff50c0c2e80 0x7ff50c0c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.693+0000 7ff50b400640 1 -- 172.21.15.47:0/774843411 >> v1:172.21.15.47:6789/0 conn(0x7ff50c0c9c80 legacy=0x7ff50c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.693+0000 7ff50b400640 1 -- 172.21.15.47:0/774843411 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c15ae20 con 0x7ff50c143a60 2024-04-24T22:42:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.694+0000 7ff50b400640 1 --2- 172.21.15.47:0/774843411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c143a60 0x7ff50c15a500 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009870 tx=0x7ff4f802a310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9b706ce579a7d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.694+0000 7ff50a000640 1 -- 172.21.15.47:0/774843411 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8037070 con 0x7ff50c143a60 2024-04-24T22:42:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.695+0000 7ff50a000640 1 -- 172.21.15.47:0/774843411 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f8032640 con 0x7ff50c143a60 2024-04-24T22:42:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.695+0000 7ff50a000640 1 -- 172.21.15.47:0/774843411 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f802ae60 con 0x7ff50c143a60 2024-04-24T22:42:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.695+0000 7ff512400640 1 -- 172.21.15.47:0/774843411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c143a60 msgr2=0x7ff50c15a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.695+0000 7ff512400640 1 --2- 172.21.15.47:0/774843411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c143a60 0x7ff50c15a500 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009870 tx=0x7ff4f802a310 comp rx=0 tx=0).stop 2024-04-24T22:42:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.695+0000 7ff512400640 1 -- 172.21.15.47:0/774843411 shutdown_connections 2024-04-24T22:42:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.695+0000 7ff512400640 1 --2- 172.21.15.47:0/774843411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff50c0c2e80 0x7ff50c0c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.695+0000 7ff512400640 1 --2- 172.21.15.47:0/774843411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c143a60 0x7ff50c15a500 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.695+0000 7ff512400640 1 -- 172.21.15.47:0/774843411 >> 172.21.15.47:0/774843411 conn(0x7ff50c073460 msgr2=0x7ff50c070730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.696+0000 7ff512400640 1 -- 172.21.15.47:0/774843411 shutdown_connections 2024-04-24T22:42:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.696+0000 7ff512400640 1 -- 172.21.15.47:0/774843411 wait complete. 2024-04-24T22:42:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.697+0000 7ff512400640 1 Processor -- start 2024-04-24T22:42:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.697+0000 7ff512400640 1 -- start start 2024-04-24T22:42:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.698+0000 7ff512400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c0c2e80 0x7ff50c155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.698+0000 7ff512400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff50c0c9c80 0x7ff50c0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.698+0000 7ff512400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff50c143a60 0x7ff50c1f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.698+0000 7ff512400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c0c5410 con 0x7ff50c143a60 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.698+0000 7ff512400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c0c5130 con 0x7ff50c0c2e80 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.698+0000 7ff512400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c0cad30 con 0x7ff50c0c9c80 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.699+0000 7ff50b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c0c2e80 0x7ff50c155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.699+0000 7ff50b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c0c2e80 0x7ff50c155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60362/0 (socket says 172.21.15.47:60362) 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.699+0000 7ff50b400640 1 -- 172.21.15.47:0/3523760926 learned_addr learned my addr 172.21.15.47:0/3523760926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.699+0000 7ff50be00640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff50c143a60 0x7ff50c1f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.699+0000 7ff50aa00640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff50c0c9c80 0x7ff50c0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.699+0000 7ff50b400640 1 -- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff50c0c9c80 msgr2=0x7ff50c0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.699+0000 7ff50b400640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff50c0c9c80 0x7ff50c0ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.699+0000 7ff50b400640 1 -- 172.21.15.47:0/3523760926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff50c143a60 msgr2=0x7ff50c1f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.699+0000 7ff50b400640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff50c143a60 0x7ff50c1f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.699+0000 7ff50b400640 1 -- 172.21.15.47:0/3523760926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c1f8520 con 0x7ff50c0c2e80 2024-04-24T22:42:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.700+0000 7ff50b400640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c0c2e80 0x7ff50c155600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8000c00 tx=0x7ff4f80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.700+0000 7ff503400640 1 -- 172.21.15.47:0/3523760926 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8052020 con 0x7ff50c0c2e80 2024-04-24T22:42:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.700+0000 7ff503400640 1 -- 172.21.15.47:0/3523760926 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f8030800 con 0x7ff50c0c2e80 2024-04-24T22:42:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.700+0000 7ff503400640 1 -- 172.21.15.47:0/3523760926 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8037040 con 0x7ff50c0c2e80 2024-04-24T22:42:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.701+0000 7ff512400640 1 -- 172.21.15.47:0/3523760926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c0ccfb0 con 0x7ff50c0c2e80 2024-04-24T22:42:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.701+0000 7ff512400640 1 -- 172.21.15.47:0/3523760926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c0cd570 con 0x7ff50c0c2e80 2024-04-24T22:42:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.703+0000 7ff512400640 1 -- 172.21.15.47:0/3523760926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50c073b90 con 0x7ff50c0c2e80 2024-04-24T22:42:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.708+0000 7ff503400640 1 -- 172.21.15.47:0/3523760926 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4f8055020 con 0x7ff50c0c2e80 2024-04-24T22:42:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.711+0000 7ff503400640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff4ec06be30 0x7ff4ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:09.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.711+0000 7ff503400640 1 -- 172.21.15.47:0/3523760926 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff4f80bc770 con 0x7ff50c0c2e80 2024-04-24T22:42:09.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.711+0000 7ff503400640 1 -- 172.21.15.47:0/3523760926 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff4f805c030 con 0x7ff50c0c2e80 2024-04-24T22:42:09.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.712+0000 7ff50aa00640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff4ec06be30 0x7ff4ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:09.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:09.713+0000 7ff50aa00640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff4ec06be30 0x7ff4ec06e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff50c1f8860 tx=0x7ff4fc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.101+0000 7ff512400640 1 -- 172.21.15.47:0/3523760926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff50c067de0 con 0x7ff50c0c2e80 2024-04-24T22:42:10.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.102+0000 7ff503400640 1 -- 172.21.15.47:0/3523760926 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7ff4f808aac0 con 0x7ff50c0c2e80 2024-04-24T22:42:10.103 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:42:10.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 -- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff4ec06be30 msgr2=0x7ff4ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff4ec06be30 0x7ff4ec06e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff50c1f8860 tx=0x7ff4fc009190 comp rx=0 tx=0).stop 2024-04-24T22:42:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 -- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c0c2e80 msgr2=0x7ff50c155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c0c2e80 0x7ff50c155600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8000c00 tx=0x7ff4f80046c0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 -- 172.21.15.47:0/3523760926 shutdown_connections 2024-04-24T22:42:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff4ec06be30 0x7ff4ec06e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff50c143a60 0x7ff50c1f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff50c0c9c80 0x7ff50c0ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 --2- 172.21.15.47:0/3523760926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff50c0c2e80 0x7ff50c155600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 -- 172.21.15.47:0/3523760926 >> 172.21.15.47:0/3523760926 conn(0x7ff50c073460 msgr2=0x7ff50c141260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 -- 172.21.15.47:0/3523760926 shutdown_connections 2024-04-24T22:42:10.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.106+0000 7ff512400640 1 -- 172.21.15.47:0/3523760926 wait complete. 2024-04-24T22:42:10.125 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-24T22:42:10.125 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-24T22:42:10.125 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-24T22:42:10.125 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:42:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.418+0000 7f75f3800640 1 Processor -- start 2024-04-24T22:42:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.418+0000 7f75f3800640 1 -- start start 2024-04-24T22:42:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.419+0000 7f75f3800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.419+0000 7f75f3800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75ec072b80 0x7f75ec073010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.419+0000 7f75f3800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75ec0c9400 con 0x7f75ec0c8fa0 2024-04-24T22:42:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.419+0000 7f75f3800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f75ec158f20 con 0x7f75ec072b80 2024-04-24T22:42:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.419+0000 7f75f3800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec1593b0 con 0x7f75ec152960 2024-04-24T22:42:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.420+0000 7f75f0e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.420+0000 7f75f1800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75ec072b80 0x7f75ec073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.420+0000 7f75ebe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f75ec0c8fa0 0x7f75ec15ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36294/0 (socket says 172.21.15.47:36294) 2024-04-24T22:42:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.420+0000 7f75ebe00640 1 -- 172.21.15.47:0/3001037840 learned_addr learned my addr 172.21.15.47:0/3001037840 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:42:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.420+0000 7f75f0e00640 1 -- 172.21.15.47:0/3001037840 >> v2:172.21.15.113:3300/0 conn(0x7f75ec072b80 msgr2=0x7f75ec073010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.420+0000 7f75f0e00640 1 --2- 172.21.15.47:0/3001037840 >> v2:172.21.15.113:3300/0 conn(0x7f75ec072b80 0x7f75ec073010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.420+0000 7f75f0e00640 1 -- 172.21.15.47:0/3001037840 >> v1:172.21.15.47:6789/0 conn(0x7f75ec0c8fa0 legacy=0x7f75ec15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.420+0000 7f75f0e00640 1 -- 172.21.15.47:0/3001037840 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec0c7e50 con 0x7f75ec152960 2024-04-24T22:42:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.420+0000 7f75eb400640 1 -- 172.21.15.47:0/3001037840 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3968636446 0 0) 0x7f75ec0c9400 con 0x7f75ec0c8fa0 2024-04-24T22:42:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.420+0000 7f75f1800640 1 --2- 172.21.15.47:0/3001037840 >> v2:172.21.15.113:3300/0 conn(0x7f75ec072b80 0x7f75ec073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.421+0000 7f75f0e00640 1 --2- 172.21.15.47:0/3001037840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec0c7750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f75e00097a0 tx=0x7f75e002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bae81deece85e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75eb400640 1 -- 172.21.15.47:0/3001037840 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0037070 con 0x7f75ec152960 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75eb400640 1 -- 172.21.15.47:0/3001037840 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e0032440 con 0x7f75ec152960 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75eb400640 1 -- 172.21.15.47:0/3001037840 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0032810 con 0x7f75ec152960 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75f3800640 1 -- 172.21.15.47:0/3001037840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 msgr2=0x7f75ec0c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75f3800640 1 --2- 172.21.15.47:0/3001037840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec0c7750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f75e00097a0 tx=0x7f75e002a210 comp rx=0 tx=0).stop 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75f3800640 1 -- 172.21.15.47:0/3001037840 shutdown_connections 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75f3800640 1 --2- 172.21.15.47:0/3001037840 >> v2:172.21.15.113:3300/0 conn(0x7f75ec072b80 0x7f75ec073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75f3800640 1 --2- 172.21.15.47:0/3001037840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec0c7750 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75f3800640 1 -- 172.21.15.47:0/3001037840 >> 172.21.15.47:0/3001037840 conn(0x7f75ec06e580 msgr2=0x7f75ec06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75f3800640 1 -- 172.21.15.47:0/3001037840 shutdown_connections 2024-04-24T22:42:10.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.422+0000 7f75f3800640 1 -- 172.21.15.47:0/3001037840 wait complete. 2024-04-24T22:42:10.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.424+0000 7f75f3800640 1 Processor -- start 2024-04-24T22:42:10.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.424+0000 7f75f3800640 1 -- start start 2024-04-24T22:42:10.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.425+0000 7f75f3800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec072b80 0x7f75ec1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.425+0000 7f75f3800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75ec0c8fa0 0x7f75ec1fec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.425+0000 7f75f3800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec1ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.425+0000 7f75f3800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec1588d0 con 0x7f75ec072b80 2024-04-24T22:42:10.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.425+0000 7f75f3800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec0c8130 con 0x7f75ec0c8fa0 2024-04-24T22:42:10.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.425+0000 7f75f3800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec158bb0 con 0x7f75ec152960 2024-04-24T22:42:10.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.425+0000 7f75f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec072b80 0x7f75ec1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.425+0000 7f75f1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec1ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.426+0000 7f75f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec072b80 0x7f75ec1508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52198/0 (socket says 172.21.15.47:52198) 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.426+0000 7f75f1800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec1ff1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:38284/0 (socket says 172.21.15.47:38284) 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.426+0000 7f75f0e00640 1 -- 172.21.15.47:0/3701079242 learned_addr learned my addr 172.21.15.47:0/3701079242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.426+0000 7f75ebe00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75ec0c8fa0 0x7f75ec1fec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.426+0000 7f75f1800640 1 -- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75ec0c8fa0 msgr2=0x7f75ec1fec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.426+0000 7f75f1800640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75ec0c8fa0 0x7f75ec1fec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.426+0000 7f75f1800640 1 -- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec072b80 msgr2=0x7f75ec1508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.426+0000 7f75f1800640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec072b80 0x7f75ec1508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.426+0000 7f75f1800640 1 -- 172.21.15.47:0/3701079242 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec0cb460 con 0x7f75ec152960 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.426+0000 7f75f1800640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec1ff1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f75e400efc0 tx=0x7f75e400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.427+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4018030 con 0x7f75ec152960 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.427+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e400cd30 con 0x7f75ec152960 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.427+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4003bb0 con 0x7f75ec152960 2024-04-24T22:42:10.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.427+0000 7f75f3800640 1 -- 172.21.15.47:0/3701079242 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ec0cb180 con 0x7f75ec152960 2024-04-24T22:42:10.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.427+0000 7f75f3800640 1 -- 172.21.15.47:0/3701079242 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ec201390 con 0x7f75ec152960 2024-04-24T22:42:10.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.428+0000 7f75f3800640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec14a670 0x7f75ec1fe500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.428+0000 7f75f3800640 1 -- 172.21.15.47:0/3701079242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec14b000 con 0x7f75ec14a670 2024-04-24T22:42:10.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.428+0000 7f75f0e00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec14a670 0x7f75ec1fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.428+0000 7f75f0e00640 1 -- 172.21.15.47:0/3701079242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75ec073b90 con 0x7f75ec14a670 2024-04-24T22:42:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.429+0000 7f75f0e00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec14a670 0x7f75ec1fe500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f75e00097a0 tx=0x7f75e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.429+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0043070 con 0x7f75ec14a670 2024-04-24T22:42:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.429+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75e401b070 con 0x7f75ec152960 2024-04-24T22:42:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.430+0000 7f75e8c00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f75cc06bd60 0x7f75cc06e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.430+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f75e00093c0 con 0x7f75ec14a670 2024-04-24T22:42:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.430+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec14a670 msgr2=0x7f75ec1fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.430+0000 7f75e8c00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec14a670 0x7f75ec1fe500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f75e00097a0 tx=0x7f75e00046c0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.430+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec14a670 msgr2=0x7f75ec1fe500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.430+0000 7f75e8c00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec14a670 0x7f75ec1fe500 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f75e00097a0 tx=0x7f75e00046c0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.430+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f75e408b870 con 0x7f75ec152960 2024-04-24T22:42:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.431+0000 7f75ebe00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f75cc06bd60 0x7f75cc06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.431+0000 7f75ebe00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f75cc06bd60 0x7f75cc06e220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f75ec1503f0 tx=0x7f75d8007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.438+0000 7f75f3800640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec0cb840 0x7f75ec149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.438+0000 7f75f3800640 1 -- 172.21.15.47:0/3701079242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec070d70 con 0x7f75ec0cb840 2024-04-24T22:42:10.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.438+0000 7f75f0e00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec0cb840 0x7f75ec149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.438+0000 7f75f0e00640 1 -- 172.21.15.47:0/3701079242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f75ec0c5780 con 0x7f75ec0cb840 2024-04-24T22:42:10.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.438+0000 7f75f0e00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec0cb840 0x7f75ec149860 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f75ec04f6a0 tx=0x7f75e0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.439+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0037040 con 0x7f75ec0cb840 2024-04-24T22:42:10.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.439+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5040 (secure 0 0 0) 0x7f75e0047cd0 con 0x7f75ec0cb840 2024-04-24T22:42:10.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.439+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec0cb840 msgr2=0x7f75ec149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.439+0000 7f75e8c00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec0cb840 0x7f75ec149860 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f75ec04f6a0 tx=0x7f75e0009190 comp rx=0 tx=0).stop 2024-04-24T22:42:10.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.439+0000 7f75e8c00640 1 -- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec0cb840 msgr2=0x7f75ec149860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:10.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.439+0000 7f75e8c00640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec0cb840 0x7f75ec149860 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.442 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:42:10.442 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:10.442 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:10.442 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:42:10.442 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 20, 2024-04-24T22:42:10.442 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 70476, 2024-04-24T22:42:10.442 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:42:10.442 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:42:10.442 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 15, 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:10.443 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:10.444 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:10.445 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:10.446 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:10.447 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:42:10.448 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:42:10.449 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:10.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:42:10.451 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:42:10.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.453 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:10.454 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:10.455 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.455 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.455 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:42:10.455 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:42:10.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.439+0000 7f75f3800640 1 -- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f75cc06bd60 msgr2=0x7f75cc06e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.439+0000 7f75f3800640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f75cc06bd60 0x7f75cc06e220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f75ec1503f0 tx=0x7f75d8007d50 comp rx=0 tx=0).stop 2024-04-24T22:42:10.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.440+0000 7f75f1800640 1 -- 172.21.15.47:0/3701079242 reap_dead start 2024-04-24T22:42:10.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.440+0000 7f75f3800640 1 -- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 msgr2=0x7f75ec1ff1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.440+0000 7f75f3800640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec1ff1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f75e400efc0 tx=0x7f75e400c490 comp rx=0 tx=0).stop 2024-04-24T22:42:10.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.440+0000 7f75f3800640 1 -- 172.21.15.47:0/3701079242 shutdown_connections 2024-04-24T22:42:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.440+0000 7f75f3800640 1 --2- 172.21.15.47:0/3701079242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75ec152960 0x7f75ec1ff1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.440+0000 7f75f3800640 1 -- 172.21.15.47:0/3701079242 >> 172.21.15.47:0/3701079242 conn(0x7f75ec06e580 msgr2=0x7f75ec0c61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.440+0000 7f75f3800640 1 -- 172.21.15.47:0/3701079242 shutdown_connections 2024-04-24T22:42:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.440+0000 7f75f3800640 1 -- 172.21.15.47:0/3701079242 wait complete. 2024-04-24T22:42:10.461 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:42:10.461 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-24T22:42:10.461 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:42:10.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:42:10.520 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:42:10.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.768+0000 7febcd800640 1 Processor -- start 2024-04-24T22:42:10.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.768+0000 7febcd800640 1 -- start start 2024-04-24T22:42:10.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.772+0000 7febcd800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febc80cc840 0x7febc80ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.773+0000 7febcd800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7febc80cbde0 0x7febc80ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.773+0000 7febcd800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc815b240 con 0x7febc8073380 2024-04-24T22:42:10.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.773+0000 7febcd800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7febc815daa0 con 0x7febc80cbde0 2024-04-24T22:42:10.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.773+0000 7febcd800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7febc815dcf0 con 0x7febc80cc840 2024-04-24T22:42:10.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.773+0000 7febc6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febc80cc840 0x7febc80ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.773+0000 7febc6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febc80cc840 0x7febc80ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:38290/0 (socket says 172.21.15.47:38290) 2024-04-24T22:42:10.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.773+0000 7febc6a00640 1 -- 172.21.15.47:0/200375587 learned_addr learned my addr 172.21.15.47:0/200375587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:10.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.773+0000 7febc7400640 1 --2- 172.21.15.47:0/200375587 >> v2:172.21.15.113:3300/0 conn(0x7febc80cbde0 0x7febc80ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.774+0000 7febc6a00640 1 -- 172.21.15.47:0/200375587 >> v2:172.21.15.113:3300/0 conn(0x7febc80cbde0 msgr2=0x7febc80ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:10.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.774+0000 7febc6a00640 1 --2- 172.21.15.47:0/200375587 >> v2:172.21.15.113:3300/0 conn(0x7febc80cbde0 0x7febc80ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.774+0000 7febc6a00640 1 -- 172.21.15.47:0/200375587 >> v1:172.21.15.113:6789/0 conn(0x7febc8073380 legacy=0x7febc80cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.774+0000 7febc6a00640 1 -- 172.21.15.47:0/200375587 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc815dfd0 con 0x7febc80cc840 2024-04-24T22:42:10.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.775+0000 7febc6a00640 1 --2- 172.21.15.47:0/200375587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febc80cc840 0x7febc80ca7d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7febb8009e50 tx=0x7febb802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae136b07c164866a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.775+0000 7febc6000640 1 -- 172.21.15.47:0/200375587 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8036070 con 0x7febc80cc840 2024-04-24T22:42:10.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.775+0000 7febc6000640 1 -- 172.21.15.47:0/200375587 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb8031600 con 0x7febc80cc840 2024-04-24T22:42:10.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.775+0000 7febc6000640 1 -- 172.21.15.47:0/200375587 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb80319d0 con 0x7febc80cc840 2024-04-24T22:42:10.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.777+0000 7febcd800640 1 -- 172.21.15.47:0/200375587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febc80cc840 msgr2=0x7febc80ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.777+0000 7febcd800640 1 --2- 172.21.15.47:0/200375587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febc80cc840 0x7febc80ca7d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7febb8009e50 tx=0x7febb802a710 comp rx=0 tx=0).stop 2024-04-24T22:42:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.779+0000 7febcd800640 1 -- 172.21.15.47:0/200375587 shutdown_connections 2024-04-24T22:42:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.779+0000 7febcd800640 1 --2- 172.21.15.47:0/200375587 >> v2:172.21.15.113:3300/0 conn(0x7febc80cbde0 0x7febc80ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.779+0000 7febcd800640 1 --2- 172.21.15.47:0/200375587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febc80cc840 0x7febc80ca7d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.779+0000 7febcd800640 1 -- 172.21.15.47:0/200375587 >> 172.21.15.47:0/200375587 conn(0x7febc806c940 msgr2=0x7febc8072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:10.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.779+0000 7febcd800640 1 -- 172.21.15.47:0/200375587 shutdown_connections 2024-04-24T22:42:10.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.779+0000 7febcd800640 1 -- 172.21.15.47:0/200375587 wait complete. 2024-04-24T22:42:10.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.781+0000 7febcd800640 1 Processor -- start 2024-04-24T22:42:10.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.781+0000 7febcd800640 1 -- start start 2024-04-24T22:42:10.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.782+0000 7febcd800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febc8073380 0x7febc80ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.782+0000 7febcd800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc80cbde0 0x7febc814f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.782+0000 7febcd800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80cc840 0x7febc81500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.782+0000 7febcd800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7febc815e870 con 0x7febc80cbde0 2024-04-24T22:42:10.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.782+0000 7febcd800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7febc815e590 con 0x7febc80cc840 2024-04-24T22:42:10.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.782+0000 7febcd800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7febc815af30 con 0x7febc8073380 2024-04-24T22:42:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.783+0000 7febc7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80cc840 0x7febc81500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.783+0000 7febc7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80cc840 0x7febc81500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60380/0 (socket says 172.21.15.47:60380) 2024-04-24T22:42:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.783+0000 7febbfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc80cbde0 0x7febc814f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.783+0000 7febc7400640 1 -- 172.21.15.47:0/327028962 learned_addr learned my addr 172.21.15.47:0/327028962 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.784+0000 7febc7400640 1 -- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febc8073380 msgr2=0x7febc80ceef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T22:42:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.784+0000 7febc7400640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febc8073380 0x7febc80ceef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.784+0000 7febc7400640 1 -- 172.21.15.47:0/327028962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc80cbde0 msgr2=0x7febc814f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.784+0000 7febc7400640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc80cbde0 0x7febc814f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.784+0000 7febc7400640 1 -- 172.21.15.47:0/327028962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc814fbd0 con 0x7febc80cc840 2024-04-24T22:42:10.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.784+0000 7febc7400640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80cc840 0x7febc81500b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febc000e930 tx=0x7febc000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.784+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0017070 con 0x7febc80cc840 2024-04-24T22:42:10.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.785+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc000c980 con 0x7febc80cc840 2024-04-24T22:42:10.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.785+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc000cd70 con 0x7febc80cc840 2024-04-24T22:42:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.785+0000 7febcd800640 1 -- 172.21.15.47:0/327028962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc8151b50 con 0x7febc80cc840 2024-04-24T22:42:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.785+0000 7febcd800640 1 -- 172.21.15.47:0/327028962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc80ce170 con 0x7febc80cc840 2024-04-24T22:42:10.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.787+0000 7febcd800640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80ce370 0x7febc81505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.787+0000 7febcd800640 1 -- 172.21.15.47:0/327028962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7febc80cf430 con 0x7febc80ce370 2024-04-24T22:42:10.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.788+0000 7febc6a00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80ce370 0x7febc81505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.788+0000 7febc6a00640 1 -- 172.21.15.47:0/327028962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febc8154a10 con 0x7febc80ce370 2024-04-24T22:42:10.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.788+0000 7febc6a00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80ce370 0x7febc81505f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7febc80d0f50 tx=0x7febb8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.790+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8036050 con 0x7febc80ce370 2024-04-24T22:42:10.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.790+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7febb8037040 con 0x7febc80ce370 2024-04-24T22:42:10.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.790+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80ce370 msgr2=0x7febc81505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.790+0000 7febbea00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80ce370 0x7febc81505f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7febc80d0f50 tx=0x7febb8004a40 comp rx=0 tx=0).stop 2024-04-24T22:42:10.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.790+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80ce370 msgr2=0x7febc81505f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:10.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.790+0000 7febbea00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80ce370 0x7febc81505f0 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7febc80d0f50 tx=0x7febb8004a40 comp rx=0 tx=0).stop 2024-04-24T22:42:10.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.794+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febc001a070 con 0x7febc80cc840 2024-04-24T22:42:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.796+0000 7febbea00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feba406b960 0x7feba406de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.796+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7febc008b060 con 0x7febc80cc840 2024-04-24T22:42:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.797+0000 7febc6a00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feba406b960 0x7feba406de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.798+0000 7febc6a00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feba406b960 0x7feba406de20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7febc806c4a0 tx=0x7febb803b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.810+0000 7febcd800640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc8151680 0x7febc8146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.811+0000 7febcd800640 1 -- 172.21.15.47:0/327028962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7febc814fdd0 con 0x7febc8151680 2024-04-24T22:42:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.811+0000 7febbfe00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc8151680 0x7febc8146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.812+0000 7febbfe00640 1 -- 172.21.15.47:0/327028962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7febc8073b90 con 0x7febc8151680 2024-04-24T22:42:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.812+0000 7febbfe00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc8151680 0x7febc8146d30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7febc8071640 tx=0x7febb000f9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.813+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb00052e0 con 0x7febc8151680 2024-04-24T22:42:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.813+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5066 (secure 0 0 0) 0x7febb00052e0 con 0x7febc8151680 2024-04-24T22:42:10.815 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:42:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:42:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 20, 2024-04-24T22:42:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 172380, 2024-04-24T22:42:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:42:10.815 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 15, 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:10.816 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:10.817 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:42:10.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:10.819 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:10.821 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:10.823 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:42:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:42:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:10.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:42:10.826 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:42:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:42:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:42:10.827 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:42:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:42:10.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:42:10.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:10.830 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:42:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-24T22:42:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:10.831 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:42:10.832 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:42:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.814+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc8151680 msgr2=0x7febc8146d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.814+0000 7febbea00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc8151680 0x7febc8146d30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7febc8071640 tx=0x7febb000f9d0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.814+0000 7febbea00640 1 -- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc8151680 msgr2=0x7febc8146d30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.814+0000 7febbea00640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc8151680 0x7febc8146d30 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7febc8071640 tx=0x7febb000f9d0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.825+0000 7febcd800640 1 -- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feba406b960 msgr2=0x7feba406de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.825+0000 7febcd800640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feba406b960 0x7feba406de20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7febc806c4a0 tx=0x7febb803b000 comp rx=0 tx=0).stop 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.826+0000 7febcd800640 1 -- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80cc840 msgr2=0x7febc81500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.826+0000 7febcd800640 1 --2- 172.21.15.47:0/327028962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc80cc840 0x7febc81500b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febc000e930 tx=0x7febc000edf0 comp rx=0 tx=0).stop 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.826+0000 7febc7400640 1 -- 172.21.15.47:0/327028962 reap_dead start 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.826+0000 7febc7400640 1 -- 172.21.15.47:0/327028962 reap_dead start 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.828+0000 7febcd800640 1 -- 172.21.15.47:0/327028962 shutdown_connections 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.828+0000 7febcd800640 1 -- 172.21.15.47:0/327028962 >> 172.21.15.47:0/327028962 conn(0x7febc806c940 msgr2=0x7febc81e3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.829+0000 7febcd800640 1 -- 172.21.15.47:0/327028962 shutdown_connections 2024-04-24T22:42:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:10.830+0000 7febcd800640 1 -- 172.21.15.47:0/327028962 wait complete. 2024-04-24T22:42:10.855 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:42:10.855 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-24T22:42:10.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-24T22:42:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.183+0000 7fc99b600640 1 Processor -- start 2024-04-24T22:42:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.184+0000 7fc99b600640 1 -- start start 2024-04-24T22:42:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.186+0000 7fc99b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940ca080 0x7fc99406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.186+0000 7fc99b600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc99415a5e0 con 0x7fc9940c3b10 2024-04-24T22:42:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.187+0000 7fc993e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc9940c3b10 0x7fc99415de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36304/0 (socket says 172.21.15.47:36304) 2024-04-24T22:42:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.187+0000 7fc993e00640 1 -- 172.21.15.47:0/4021703536 learned_addr learned my addr 172.21.15.47:0/4021703536 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:42:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.187+0000 7fc999600640 1 --2- 172.21.15.47:0/4021703536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940ca080 0x7fc99406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.187+0000 7fc99b600640 1 -- 172.21.15.47:0/4021703536 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc99415a8c0 con 0x7fc9940c9ca0 2024-04-24T22:42:11.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.187+0000 7fc99b600640 1 -- 172.21.15.47:0/4021703536 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc99406c800 con 0x7fc9940ca080 2024-04-24T22:42:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.188+0000 7fc993400640 1 -- 172.21.15.47:0/4021703536 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3390739968 0 0) 0x7fc99415a5e0 con 0x7fc9940c3b10 2024-04-24T22:42:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.188+0000 7fc993400640 1 -- 172.21.15.47:0/4021703536 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc97c003520 con 0x7fc9940c3b10 2024-04-24T22:42:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.188+0000 7fc999600640 1 -- 172.21.15.47:0/4021703536 >> v1:172.21.15.113:6789/0 conn(0x7fc9940c9ca0 legacy=0x7fc9940c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.189+0000 7fc993400640 1 -- 172.21.15.47:0/4021703536 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3152415822 0 0) 0x7fc97c003520 con 0x7fc9940c3b10 2024-04-24T22:42:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.186+0000 7fc998c00640 1 --1- 172.21.15.47:0/4021703536 >> v1:172.21.15.113:6789/0 conn(0x7fc9940c9ca0 0x7fc9940c3400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:54900/0 (socket says 172.21.15.47:54900) 2024-04-24T22:42:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.189+0000 7fc999600640 1 -- 172.21.15.47:0/4021703536 >> v1:172.21.15.47:6789/0 conn(0x7fc9940c3b10 legacy=0x7fc99415de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.189+0000 7fc999600640 1 -- 172.21.15.47:0/4021703536 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99415aba0 con 0x7fc9940ca080 2024-04-24T22:42:11.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.191+0000 7fc999600640 1 --2- 172.21.15.47:0/4021703536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940ca080 0x7fc99406c2c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009dc0 tx=0x7fc98c02a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ead5ade93c9a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:11.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.191+0000 7fc993400640 1 -- 172.21.15.47:0/4021703536 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980001670 con 0x7fc9940ca080 2024-04-24T22:42:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.191+0000 7fc993400640 1 -- 172.21.15.47:0/4021703536 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c007540 con 0x7fc9940ca080 2024-04-24T22:42:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.191+0000 7fc993400640 1 -- 172.21.15.47:0/4021703536 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c007710 con 0x7fc9940ca080 2024-04-24T22:42:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.192+0000 7fc99b600640 1 -- 172.21.15.47:0/4021703536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940ca080 msgr2=0x7fc99406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.192+0000 7fc99b600640 1 --2- 172.21.15.47:0/4021703536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940ca080 0x7fc99406c2c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009dc0 tx=0x7fc98c02a6a0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.192+0000 7fc99b600640 1 -- 172.21.15.47:0/4021703536 shutdown_connections 2024-04-24T22:42:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.192+0000 7fc99b600640 1 --2- 172.21.15.47:0/4021703536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940ca080 0x7fc99406c2c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.192+0000 7fc99b600640 1 -- 172.21.15.47:0/4021703536 >> 172.21.15.47:0/4021703536 conn(0x7fc99406fb00 msgr2=0x7fc99406ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.192+0000 7fc99b600640 1 -- 172.21.15.47:0/4021703536 shutdown_connections 2024-04-24T22:42:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.192+0000 7fc99b600640 1 -- 172.21.15.47:0/4021703536 wait complete. 2024-04-24T22:42:11.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.194+0000 7fc99b600640 1 Processor -- start 2024-04-24T22:42:11.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.194+0000 7fc99b600640 1 -- start start 2024-04-24T22:42:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.195+0000 7fc99b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9940c3b10 0x7fc9941e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.195+0000 7fc99b600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940c9ca0 0x7fc9941e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.195+0000 7fc99b600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9940ca080 0x7fc9941e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.195+0000 7fc99b600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc99415e550 con 0x7fc9940ca080 2024-04-24T22:42:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.195+0000 7fc99b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc99415b160 con 0x7fc9940c3b10 2024-04-24T22:42:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.195+0000 7fc99b600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc99415e6d0 con 0x7fc9940c9ca0 2024-04-24T22:42:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.196+0000 7fc999600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9940ca080 0x7fc9941e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.196+0000 7fc999600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9940ca080 0x7fc9941e6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52254/0 (socket says 172.21.15.47:52254) 2024-04-24T22:42:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.196+0000 7fc999600640 1 -- 172.21.15.47:0/1628017199 learned_addr learned my addr 172.21.15.47:0/1628017199 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:11.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.196+0000 7fc998c00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9940c3b10 0x7fc9941e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.196+0000 7fc993e00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940c9ca0 0x7fc9941e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.197+0000 7fc999600640 1 -- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940c9ca0 msgr2=0x7fc9941e42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.197+0000 7fc999600640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940c9ca0 0x7fc9941e42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.197+0000 7fc999600640 1 -- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9940c3b10 msgr2=0x7fc9941e4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.197+0000 7fc999600640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9940c3b10 0x7fc9941e4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.197+0000 7fc999600640 1 -- 172.21.15.47:0/1628017199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9941551d0 con 0x7fc9940ca080 2024-04-24T22:42:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.197+0000 7fc999600640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9940ca080 0x7fc9941e6b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009ee0 tx=0x7fc98c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.198+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c040070 con 0x7fc9940ca080 2024-04-24T22:42:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.198+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c03c830 con 0x7fc9940ca080 2024-04-24T22:42:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.198+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c007e90 con 0x7fc9940ca080 2024-04-24T22:42:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.198+0000 7fc99b600640 1 -- 172.21.15.47:0/1628017199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9941554b0 con 0x7fc9940ca080 2024-04-24T22:42:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.198+0000 7fc99b600640 1 -- 172.21.15.47:0/1628017199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc994155a70 con 0x7fc9940ca080 2024-04-24T22:42:11.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.200+0000 7fc99b600640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc994156550 0x7fc994155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.200+0000 7fc99b600640 1 -- 172.21.15.47:0/1628017199 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc99415de20 con 0x7fc994156550 2024-04-24T22:42:11.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.201+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc98c039040 con 0x7fc9940ca080 2024-04-24T22:42:11.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.201+0000 7fc998c00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc994156550 0x7fc994155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.202+0000 7fc998c00640 1 -- 172.21.15.47:0/1628017199 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc994073b90 con 0x7fc994156550 2024-04-24T22:42:11.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.202+0000 7fc998c00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc994156550 0x7fc994155c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc98800e700 tx=0x7fc98800ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:11.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.203+0000 7fc990c00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc97406be80 0x7fc97406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.203+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc98c0ac430 con 0x7fc9940ca080 2024-04-24T22:42:11.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.203+0000 7fc993e00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc97406be80 0x7fc97406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.203+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98801a070 con 0x7fc994156550 2024-04-24T22:42:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.203+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc988004b40 con 0x7fc994156550 2024-04-24T22:42:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.203+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc994156550 msgr2=0x7fc994155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.203+0000 7fc990c00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc994156550 0x7fc994155c70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc98800e700 tx=0x7fc98800ebc0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.203+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc994156550 msgr2=0x7fc994155c70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.203+0000 7fc990c00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc994156550 0x7fc994155c70 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc98800e700 tx=0x7fc98800ebc0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.204+0000 7fc993e00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc97406be80 0x7fc97406e340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc9941e67b0 tx=0x7fc980009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:11.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.212+0000 7fc99b600640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940b2b50 0x7fc99414f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.212+0000 7fc99b600640 1 -- 172.21.15.47:0/1628017199 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc9940c7b60 con 0x7fc9940b2b50 2024-04-24T22:42:11.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.213+0000 7fc998c00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940b2b50 0x7fc99414f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.213+0000 7fc998c00640 1 -- 172.21.15.47:0/1628017199 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc9941e5010 con 0x7fc9940b2b50 2024-04-24T22:42:11.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.213+0000 7fc998c00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940b2b50 0x7fc99414f300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc988017000 tx=0x7fc988004930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:11.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.214+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98801a040 con 0x7fc9940b2b50 2024-04-24T22:42:11.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.214+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4555 (secure 0 0 0) 0x7fc98800c6b0 con 0x7fc9940b2b50 2024-04-24T22:42:11.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.214+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940b2b50 msgr2=0x7fc99414f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.214+0000 7fc990c00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940b2b50 0x7fc99414f300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc988017000 tx=0x7fc988004930 comp rx=0 tx=0).stop 2024-04-24T22:42:11.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.214+0000 7fc990c00640 1 -- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940b2b50 msgr2=0x7fc99414f300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:11.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.214+0000 7fc990c00640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9940b2b50 0x7fc99414f300 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc988017000 tx=0x7fc988004930 comp rx=0 tx=0).stop 2024-04-24T22:42:11.217 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:42:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:42:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:42:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:42:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 20, 2024-04-24T22:42:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 22862, 2024-04-24T22:42:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:42:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:42:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:42:11.217 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 16, 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:11.218 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:11.219 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:42:11.220 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:11.221 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.222 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:42:11.223 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:11.224 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:42:11.226 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:11.227 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:11.228 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:11.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:11.229 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:11.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:11.229 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:42:11.229 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:42:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.215+0000 7fc99b600640 1 -- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc97406be80 msgr2=0x7fc97406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.215+0000 7fc99b600640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc97406be80 0x7fc97406e340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc9941e67b0 tx=0x7fc980009040 comp rx=0 tx=0).stop 2024-04-24T22:42:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.215+0000 7fc99b600640 1 -- 172.21.15.47:0/1628017199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9940ca080 msgr2=0x7fc9941e6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.215+0000 7fc99b600640 1 --2- 172.21.15.47:0/1628017199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9940ca080 0x7fc9941e6b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009ee0 tx=0x7fc98c004a40 comp rx=0 tx=0).stop 2024-04-24T22:42:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.215+0000 7fc999600640 1 -- 172.21.15.47:0/1628017199 reap_dead start 2024-04-24T22:42:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.215+0000 7fc999600640 1 -- 172.21.15.47:0/1628017199 reap_dead start 2024-04-24T22:42:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.215+0000 7fc99b600640 1 -- 172.21.15.47:0/1628017199 shutdown_connections 2024-04-24T22:42:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.215+0000 7fc99b600640 1 -- 172.21.15.47:0/1628017199 >> 172.21.15.47:0/1628017199 conn(0x7fc99406fb00 msgr2=0x7fc9940c72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.215+0000 7fc99b600640 1 -- 172.21.15.47:0/1628017199 shutdown_connections 2024-04-24T22:42:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.215+0000 7fc99b600640 1 -- 172.21.15.47:0/1628017199 wait complete. 2024-04-24T22:42:11.253 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:42:11.253 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-24T22:42:11.253 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-24T22:42:11.253 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-24T22:42:11.253 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-24T22:42:11.253 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-24T22:42:11.253 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:42:11.266 INFO:tasks.ceph.mon.c:Stopped 2024-04-24T22:42:11.267 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-24T22:42:11.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:42:11.406 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:11.406 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 initial writers = 259147.80 kB/sec 2024-04-24T22:42:11.406 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 initial writers = 259098.23 kB/sec 2024-04-24T22:42:11.406 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 259147.80 kB/sec 2024-04-24T22:42:11.406 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 259147.80 kB/sec 2024-04-24T22:42:11.406 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 259147.80 kB/sec 2024-04-24T22:42:11.406 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 1048576.00 kB 2024-04-24T22:42:11.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.489+0000 7fdee6c00640 1 Processor -- start 2024-04-24T22:42:11.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.489+0000 7fdee6c00640 1 -- start start 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.490+0000 7fdee6c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdee00c1a00 0x7fdee0072f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.490+0000 7fdee6c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdee00734a0 0x7fdee01576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.490+0000 7fdee6c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee00738f0 con 0x7fdee0151780 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.490+0000 7fdee6c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdee015aa30 con 0x7fdee00734a0 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.490+0000 7fdee6c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdee015ad10 con 0x7fdee00c1a00 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdedfe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdee0151780 0x7fdee00c8590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36310/0 (socket says 172.21.15.47:36310) 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdedfe00640 1 -- 172.21.15.47:0/4128160051 learned_addr learned my addr 172.21.15.47:0/4128160051 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdee4c00640 1 --2- 172.21.15.47:0/4128160051 >> v2:172.21.15.47:3300/0 conn(0x7fdee00734a0 0x7fdee01576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdedf400640 1 --2- 172.21.15.47:0/4128160051 >> v2:172.21.15.113:3300/0 conn(0x7fdee00c1a00 0x7fdee0072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdedea00640 1 -- 172.21.15.47:0/4128160051 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152395710 0 0) 0x7fded0001110 con 0x7fdee0151780 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdedea00640 1 -- 172.21.15.47:0/4128160051 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdec4003520 con 0x7fdee0151780 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdee4c00640 1 -- 172.21.15.47:0/4128160051 >> v2:172.21.15.113:3300/0 conn(0x7fdee00c1a00 msgr2=0x7fdee0072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:11.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdee4c00640 1 --2- 172.21.15.47:0/4128160051 >> v2:172.21.15.113:3300/0 conn(0x7fdee00c1a00 0x7fdee0072f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdee4c00640 1 -- 172.21.15.47:0/4128160051 >> v1:172.21.15.47:6789/0 conn(0x7fdee0151780 legacy=0x7fdee00c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdee4c00640 1 -- 172.21.15.47:0/4128160051 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee015aff0 con 0x7fdee00734a0 2024-04-24T22:42:11.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.491+0000 7fdedf400640 1 --2- 172.21.15.47:0/4128160051 >> v2:172.21.15.113:3300/0 conn(0x7fdee00c1a00 0x7fdee0072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:11.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.492+0000 7fdee4c00640 1 --2- 172.21.15.47:0/4128160051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00734a0 0x7fdee01576f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdee00705a0 tx=0x7fded802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948a18991a83cb22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:11.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.493+0000 7fdedea00640 1 -- 172.21.15.47:0/4128160051 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8037070 con 0x7fdee00734a0 2024-04-24T22:42:11.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.493+0000 7fdedea00640 1 -- 172.21.15.47:0/4128160051 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded8032440 con 0x7fdee00734a0 2024-04-24T22:42:11.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.493+0000 7fdedea00640 1 -- 172.21.15.47:0/4128160051 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8032810 con 0x7fdee00734a0 2024-04-24T22:42:11.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.493+0000 7fdee6c00640 1 -- 172.21.15.47:0/4128160051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00734a0 msgr2=0x7fdee01576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.493+0000 7fdee6c00640 1 --2- 172.21.15.47:0/4128160051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00734a0 0x7fdee01576f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdee00705a0 tx=0x7fded802a250 comp rx=0 tx=0).stop 2024-04-24T22:42:11.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.493+0000 7fdee6c00640 1 -- 172.21.15.47:0/4128160051 shutdown_connections 2024-04-24T22:42:11.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.493+0000 7fdee6c00640 1 --2- 172.21.15.47:0/4128160051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00734a0 0x7fdee01576f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.493+0000 7fdee6c00640 1 --2- 172.21.15.47:0/4128160051 >> v2:172.21.15.113:3300/0 conn(0x7fdee00c1a00 0x7fdee0072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.493+0000 7fdee6c00640 1 -- 172.21.15.47:0/4128160051 >> 172.21.15.47:0/4128160051 conn(0x7fdee006e580 msgr2=0x7fdee006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:11.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.493+0000 7fdee6c00640 1 -- 172.21.15.47:0/4128160051 shutdown_connections 2024-04-24T22:42:11.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.494+0000 7fdee6c00640 1 -- 172.21.15.47:0/4128160051 wait complete. 2024-04-24T22:42:11.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.495+0000 7fdee6c00640 1 Processor -- start 2024-04-24T22:42:11.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.495+0000 7fdee6c00640 1 -- start start 2024-04-24T22:42:11.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdee6c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdee00734a0 0x7fdee0146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdee6c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00c1a00 0x7fdee0158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdee6c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdee0151780 0x7fdee01589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdee6c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdee00c8ca0 con 0x7fdee00c1a00 2024-04-24T22:42:11.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdee6c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdee015b5b0 con 0x7fdee0151780 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdee6c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdee00c8e20 con 0x7fdee00734a0 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdedfe00640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdee00734a0 msgr2=0x7fdee0146460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdee00734a0 0x7fdee0146460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdedf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00c1a00 0x7fdee0158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdee4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdee0151780 0x7fdee01589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdedf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00c1a00 0x7fdee0158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52270/0 (socket says 172.21.15.47:52270) 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.496+0000 7fdedf400640 1 -- 172.21.15.47:0/159858619 learned_addr learned my addr 172.21.15.47:0/159858619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fdee4c00640 1 -- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdee00734a0 msgr2=0x7fdee0146460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fdee4c00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdee00734a0 0x7fdee0146460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fdee4c00640 1 -- 172.21.15.47:0/159858619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00c1a00 msgr2=0x7fdee0158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fdee4c00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00c1a00 0x7fdee0158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fdee4c00640 1 -- 172.21.15.47:0/159858619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee00aec90 con 0x7fdee0151780 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fdedf400640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00c1a00 0x7fdee0158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:11.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fdee4c00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdee0151780 0x7fdee01589a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fded8031000 tx=0x7fded80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:11.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fded7e00640 1 -- 172.21.15.47:0/159858619 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8041030 con 0x7fdee0151780 2024-04-24T22:42:11.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fded7e00640 1 -- 172.21.15.47:0/159858619 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded803d4e0 con 0x7fdee0151780 2024-04-24T22:42:11.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fded7e00640 1 -- 172.21.15.47:0/159858619 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8037040 con 0x7fdee0151780 2024-04-24T22:42:11.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.497+0000 7fdee6c00640 1 -- 172.21.15.47:0/159858619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee01f3c40 con 0x7fdee0151780 2024-04-24T22:42:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.498+0000 7fdee6c00640 1 -- 172.21.15.47:0/159858619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee01f4200 con 0x7fdee0151780 2024-04-24T22:42:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.498+0000 7fdee6c00640 1 -- 172.21.15.47:0/159858619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee01f47f0 con 0x7fdee0151780 2024-04-24T22:42:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.500+0000 7fded7e00640 1 -- 172.21.15.47:0/159858619 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fded8004030 con 0x7fdee0151780 2024-04-24T22:42:11.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.501+0000 7fded7e00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fdec006be10 0x7fdec006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:11.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.501+0000 7fded7e00640 1 -- 172.21.15.47:0/159858619 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fded80aaea0 con 0x7fdee0151780 2024-04-24T22:42:11.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.501+0000 7fdedfe00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fdec006be10 0x7fdec006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:11.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.502+0000 7fdedfe00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fdec006be10 0x7fdec006e2d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fded0000c00 tx=0x7fded0001230 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.504+0000 7fded7e00640 1 -- 172.21.15.47:0/159858619 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fded80797b0 con 0x7fdee0151780 2024-04-24T22:42:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.771+0000 7fdee6c00640 1 -- 172.21.15.47:0/159858619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdee0073b90 con 0x7fdee0151780 2024-04-24T22:42:11.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.772+0000 7fded7e00640 1 -- 172.21.15.47:0/159858619 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fded80790b0 con 0x7fdee0151780 2024-04-24T22:42:11.773 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:42:11.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.774+0000 7fdee6c00640 1 -- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fdec006be10 msgr2=0x7fdec006e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.774+0000 7fdee6c00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fdec006be10 0x7fdec006e2d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fded0000c00 tx=0x7fded0001230 comp rx=0 tx=0).stop 2024-04-24T22:42:11.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.774+0000 7fdee6c00640 1 -- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdee0151780 msgr2=0x7fdee01589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:11.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.774+0000 7fdee6c00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdee0151780 0x7fdee01589a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fded8031000 tx=0x7fded80046c0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.774+0000 7fdee6c00640 1 -- 172.21.15.47:0/159858619 shutdown_connections 2024-04-24T22:42:11.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.774+0000 7fdee6c00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fdec006be10 0x7fdec006e2d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.774+0000 7fdee6c00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdee0151780 0x7fdee01589a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.774+0000 7fdee6c00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdee00c1a00 0x7fdee0158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.774+0000 7fdee6c00640 1 --2- 172.21.15.47:0/159858619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdee00734a0 0x7fdee0146460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.774+0000 7fdee6c00640 1 -- 172.21.15.47:0/159858619 >> 172.21.15.47:0/159858619 conn(0x7fdee006e580 msgr2=0x7fdee006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.775+0000 7fdee6c00640 1 -- 172.21.15.47:0/159858619 shutdown_connections 2024-04-24T22:42:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:11.775+0000 7fdee6c00640 1 -- 172.21.15.47:0/159858619 wait complete. 2024-04-24T22:42:14.793 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:42:15.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.039+0000 7f79c9e00640 1 Processor -- start 2024-04-24T22:42:15.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.039+0000 7f79c9e00640 1 -- start start 2024-04-24T22:42:15.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.041+0000 7f79c9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79c4152960 0x7f79c40c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:15.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.041+0000 7f79c9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c40c8fa0 0x7f79c415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:15.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.041+0000 7f79c9e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c40c9400 con 0x7f79c415bf60 2024-04-24T22:42:15.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.041+0000 7f79c9e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f79c4158f20 con 0x7f79c4152960 2024-04-24T22:42:15.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.041+0000 7f79c9e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f79c41593b0 con 0x7f79c40c8fa0 2024-04-24T22:42:15.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.042+0000 7f79c2000640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c40c8fa0 msgr2=0x7f79c415ba20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:15.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.042+0000 7f79c2000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c40c8fa0 0x7f79c415ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:15.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.042+0000 7f79c2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79c4152960 0x7f79c40c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:15.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.042+0000 7f79c3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f79c415bf60 0x7f79c4072b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:54902/0 (socket says 172.21.15.47:54902) 2024-04-24T22:42:15.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.042+0000 7f79c3400640 1 -- 172.21.15.47:0/2231138599 learned_addr learned my addr 172.21.15.47:0/2231138599 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:42:15.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.042+0000 7f79c2a00640 1 -- 172.21.15.47:0/2231138599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c40c8fa0 msgr2=0x7f79c415ba20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:15.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.042+0000 7f79c2a00640 1 --2- 172.21.15.47:0/2231138599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c40c8fa0 0x7f79c415ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.042+0000 7f79c2a00640 1 -- 172.21.15.47:0/2231138599 >> v1:172.21.15.113:6789/0 conn(0x7f79c415bf60 legacy=0x7f79c4072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:15.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.042+0000 7f79c2a00640 1 -- 172.21.15.47:0/2231138599 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c40c7e50 con 0x7f79c4152960 2024-04-24T22:42:15.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.044+0000 7f79c2a00640 1 --2- 172.21.15.47:0/2231138599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c4152960 0x7f79c40c7750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f79b40097a0 tx=0x7f79b402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=671dc4c78a4eaa2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:15.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.044+0000 7f79c1600640 1 -- 172.21.15.47:0/2231138599 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4037070 con 0x7f79c4152960 2024-04-24T22:42:15.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.044+0000 7f79c1600640 1 -- 172.21.15.47:0/2231138599 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b4032440 con 0x7f79c4152960 2024-04-24T22:42:15.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.044+0000 7f79c1600640 1 -- 172.21.15.47:0/2231138599 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4032810 con 0x7f79c4152960 2024-04-24T22:42:15.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.045+0000 7f79c9e00640 1 -- 172.21.15.47:0/2231138599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c4152960 msgr2=0x7f79c40c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:15.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.045+0000 7f79c9e00640 1 --2- 172.21.15.47:0/2231138599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c4152960 0x7f79c40c7750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f79b40097a0 tx=0x7f79b402a230 comp rx=0 tx=0).stop 2024-04-24T22:42:15.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.045+0000 7f79c9e00640 1 -- 172.21.15.47:0/2231138599 shutdown_connections 2024-04-24T22:42:15.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.045+0000 7f79c9e00640 1 --2- 172.21.15.47:0/2231138599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c40c8fa0 0x7f79c415ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.045+0000 7f79c9e00640 1 --2- 172.21.15.47:0/2231138599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c4152960 0x7f79c40c7750 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.045+0000 7f79c9e00640 1 -- 172.21.15.47:0/2231138599 >> 172.21.15.47:0/2231138599 conn(0x7f79c406e580 msgr2=0x7f79c406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:15.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.045+0000 7f79c9e00640 1 -- 172.21.15.47:0/2231138599 shutdown_connections 2024-04-24T22:42:15.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.045+0000 7f79c9e00640 1 -- 172.21.15.47:0/2231138599 wait complete. 2024-04-24T22:42:15.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.048+0000 7f79c9e00640 1 Processor -- start 2024-04-24T22:42:15.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.048+0000 7f79c9e00640 1 -- start start 2024-04-24T22:42:15.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.049+0000 7f79c9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c40c8fa0 0x7f79c4157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:15.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.049+0000 7f79c9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c4152960 0x7f79c40c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:15.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.050+0000 7f79c9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c415bf60 0x7f79c40c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:15.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.050+0000 7f79c9e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41588d0 con 0x7f79c415bf60 2024-04-24T22:42:15.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.050+0000 7f79c9e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f79c40c8130 con 0x7f79c40c8fa0 2024-04-24T22:42:15.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.050+0000 7f79c9e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f79c4158bb0 con 0x7f79c4152960 2024-04-24T22:42:15.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.050+0000 7f79c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c40c8fa0 0x7f79c4157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:15.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.050+0000 7f79c2000640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c4152960 msgr2=0x7f79c40c4090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:15.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.050+0000 7f79c2000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c4152960 0x7f79c40c4090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:15.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.050+0000 7f79c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c40c8fa0 0x7f79c4157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60438/0 (socket says 172.21.15.47:60438) 2024-04-24T22:42:15.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.050+0000 7f79c2a00640 1 -- 172.21.15.47:0/1342371011 learned_addr learned my addr 172.21.15.47:0/1342371011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:15.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.050+0000 7f79c3400640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c415bf60 0x7f79c40c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:15.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.051+0000 7f79c2a00640 1 -- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c4152960 msgr2=0x7f79c40c4090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:42:15.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.051+0000 7f79c2a00640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c4152960 0x7f79c40c4090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.051+0000 7f79c2a00640 1 -- 172.21.15.47:0/1342371011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c415bf60 msgr2=0x7f79c40c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:15.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.051+0000 7f79c2a00640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c415bf60 0x7f79c40c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.051+0000 7f79c2a00640 1 -- 172.21.15.47:0/1342371011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c41f4490 con 0x7f79c40c8fa0 2024-04-24T22:42:15.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.051+0000 7f79c2a00640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c40c8fa0 0x7f79c4157f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f79b402a7a0 tx=0x7f79b40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:15.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.052+0000 7f79bb400640 1 -- 172.21.15.47:0/1342371011 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4041070 con 0x7f79c40c8fa0 2024-04-24T22:42:15.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.052+0000 7f79bb400640 1 -- 172.21.15.47:0/1342371011 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b403c870 con 0x7f79c40c8fa0 2024-04-24T22:42:15.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.052+0000 7f79c9e00640 1 -- 172.21.15.47:0/1342371011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c41f4700 con 0x7f79c40c8fa0 2024-04-24T22:42:15.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.052+0000 7f79bb400640 1 -- 172.21.15.47:0/1342371011 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4037040 con 0x7f79c40c8fa0 2024-04-24T22:42:15.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.052+0000 7f79c9e00640 1 -- 172.21.15.47:0/1342371011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c40c3890 con 0x7f79c40c8fa0 2024-04-24T22:42:15.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.053+0000 7f79c9e00640 1 -- 172.21.15.47:0/1342371011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c4073b90 con 0x7f79c40c8fa0 2024-04-24T22:42:15.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.057+0000 7f79bb400640 1 -- 172.21.15.47:0/1342371011 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79b4044070 con 0x7f79c40c8fa0 2024-04-24T22:42:15.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.060+0000 7f79bb400640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79a406bde0 0x7f79a406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.060+0000 7f79bb400640 1 -- 172.21.15.47:0/1342371011 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f79b40abd60 con 0x7f79c40c8fa0 2024-04-24T22:42:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.060+0000 7f79c2000640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79a406bde0 0x7f79a406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.060+0000 7f79bb400640 1 -- 172.21.15.47:0/1342371011 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f79b40ac1e0 con 0x7f79c40c8fa0 2024-04-24T22:42:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.061+0000 7f79c2000640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79a406bde0 0x7f79a406e2a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f79b0006fd0 tx=0x7f79b0006d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.482+0000 7f79c9e00640 1 -- 172.21.15.47:0/1342371011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f79c4162b90 con 0x7f79c40c8fa0 2024-04-24T22:42:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.482+0000 7f79bb400640 1 -- 172.21.15.47:0/1342371011 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f79b407a0b0 con 0x7f79c40c8fa0 2024-04-24T22:42:15.484 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:42:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 -- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79a406bde0 msgr2=0x7f79a406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79a406bde0 0x7f79a406e2a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f79b0006fd0 tx=0x7f79b0006d80 comp rx=0 tx=0).stop 2024-04-24T22:42:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 -- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c40c8fa0 msgr2=0x7f79c4157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c40c8fa0 0x7f79c4157f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f79b402a7a0 tx=0x7f79b40046c0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 -- 172.21.15.47:0/1342371011 shutdown_connections 2024-04-24T22:42:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79a406bde0 0x7f79a406e2a0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f79b0006fd0 tx=0x7f79b0006d80 comp rx=0 tx=0).stop 2024-04-24T22:42:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79c415bf60 0x7f79c40c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79c4152960 0x7f79c40c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 --2- 172.21.15.47:0/1342371011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79c40c8fa0 0x7f79c4157f70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 -- 172.21.15.47:0/1342371011 >> 172.21.15.47:0/1342371011 conn(0x7f79c406e580 msgr2=0x7f79c40c9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 -- 172.21.15.47:0/1342371011 shutdown_connections 2024-04-24T22:42:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:15.485+0000 7f79c9e00640 1 -- 172.21.15.47:0/1342371011 wait complete. 2024-04-24T22:42:17.129 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:17.129 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 rewriters = 303099.12 kB/sec 2024-04-24T22:42:17.129 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 rewriters = 303016.54 kB/sec 2024-04-24T22:42:17.129 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 303099.12 kB/sec 2024-04-24T22:42:17.129 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 303099.12 kB/sec 2024-04-24T22:42:17.129 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 303099.12 kB/sec 2024-04-24T22:42:17.129 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 1048576.00 kB 2024-04-24T22:42:18.506 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:42:18.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.722+0000 7f1db7e00640 1 Processor -- start 2024-04-24T22:42:18.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.722+0000 7f1db7e00640 1 -- start start 2024-04-24T22:42:18.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.724+0000 7f1db7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1db80ca770 0x7f1db806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:18.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.724+0000 7f1db7e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db80c5c30 con 0x7f1db81581b0 2024-04-24T22:42:18.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.724+0000 7f1db7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db80c4eb0 con 0x7f1db80cab50 2024-04-24T22:42:18.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.724+0000 7f1db7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1db80c57a0 con 0x7f1db80ca770 2024-04-24T22:42:18.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.724+0000 7f1db7400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1db81581b0 0x7f1db8141110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54548/0 (socket says 172.21.15.47:54548) 2024-04-24T22:42:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.724+0000 7f1db7400640 1 -- 172.21.15.47:0/3913998956 learned_addr learned my addr 172.21.15.47:0/3913998956 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:42:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.724+0000 7f1db6a00640 1 --2- 172.21.15.47:0/3913998956 >> v2:172.21.15.113:3300/0 conn(0x7f1db80ca770 0x7f1db806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.725+0000 7f1db6a00640 1 -- 172.21.15.47:0/3913998956 >> v1:172.21.15.113:6789/0 conn(0x7f1db80cab50 legacy=0x7f1db80c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.725+0000 7f1db6a00640 1 -- 172.21.15.47:0/3913998956 >> v1:172.21.15.47:6789/0 conn(0x7f1db81581b0 legacy=0x7f1db8141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.725+0000 7f1db6a00640 1 -- 172.21.15.47:0/3913998956 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db80c8c70 con 0x7f1db80ca770 2024-04-24T22:42:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.726+0000 7f1db6a00640 1 --2- 172.21.15.47:0/3913998956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80ca770 0x7f1db806e000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1db80c5920 tx=0x7f1da802a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21f8bdb29af9d33e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.727+0000 7f1db5600640 1 -- 172.21.15.47:0/3913998956 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8037070 con 0x7f1db80ca770 2024-04-24T22:42:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.727+0000 7f1db5600640 1 -- 172.21.15.47:0/3913998956 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da8032440 con 0x7f1db80ca770 2024-04-24T22:42:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.727+0000 7f1db5600640 1 -- 172.21.15.47:0/3913998956 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8032810 con 0x7f1db80ca770 2024-04-24T22:42:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.727+0000 7f1db7e00640 1 -- 172.21.15.47:0/3913998956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80ca770 msgr2=0x7f1db806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.727+0000 7f1db7e00640 1 --2- 172.21.15.47:0/3913998956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80ca770 0x7f1db806e000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1db80c5920 tx=0x7f1da802a2e0 comp rx=0 tx=0).stop 2024-04-24T22:42:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.727+0000 7f1db7e00640 1 -- 172.21.15.47:0/3913998956 shutdown_connections 2024-04-24T22:42:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.727+0000 7f1db7e00640 1 --2- 172.21.15.47:0/3913998956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80ca770 0x7f1db806e000 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.727+0000 7f1db7e00640 1 -- 172.21.15.47:0/3913998956 >> 172.21.15.47:0/3913998956 conn(0x7f1db8072c40 msgr2=0x7f1db8073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.728+0000 7f1db7e00640 1 -- 172.21.15.47:0/3913998956 shutdown_connections 2024-04-24T22:42:18.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.728+0000 7f1db7e00640 1 -- 172.21.15.47:0/3913998956 wait complete. 2024-04-24T22:42:18.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.729+0000 7f1db7e00640 1 Processor -- start 2024-04-24T22:42:18.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.729+0000 7f1db7e00640 1 -- start start 2024-04-24T22:42:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.730+0000 7f1db7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1db80ca770 0x7f1db80cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:18.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.730+0000 7f1db7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80cab50 0x7f1db8152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.730+0000 7f1db7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db81581b0 0x7f1db81530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.730+0000 7f1db7e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1db806e540 con 0x7f1db81581b0 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.730+0000 7f1db7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80c5430 con 0x7f1db80cab50 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.730+0000 7f1db7e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1db80c8f50 con 0x7f1db80ca770 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.730+0000 7f1db6a00640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1db80ca770 msgr2=0x7f1db80cc120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.730+0000 7f1db6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1db80ca770 0x7f1db80cc120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.731+0000 7f1db6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80cab50 0x7f1db8152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.731+0000 7f1db6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80cab50 0x7f1db8152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:57762/0 (socket says 172.21.15.47:57762) 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.731+0000 7f1db6000640 1 -- 172.21.15.47:0/3572017915 learned_addr learned my addr 172.21.15.47:0/3572017915 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.731+0000 7f1db7400640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db81581b0 0x7f1db81530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.731+0000 7f1db6000640 1 -- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1db80ca770 msgr2=0x7f1db80cc120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.731+0000 7f1db6000640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1db80ca770 0x7f1db80cc120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:18.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.731+0000 7f1db6000640 1 -- 172.21.15.47:0/3572017915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db81581b0 msgr2=0x7f1db81530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:18.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.731+0000 7f1db6000640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db81581b0 0x7f1db81530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:18.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.731+0000 7f1db6000640 1 -- 172.21.15.47:0/3572017915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db80bd7d0 con 0x7f1db80cab50 2024-04-24T22:42:18.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.731+0000 7f1db6000640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80cab50 0x7f1db8152bb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1da000c7c0 tx=0x7f1da000cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:18.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.732+0000 7f1daea00640 1 -- 172.21.15.47:0/3572017915 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0016020 con 0x7f1db80cab50 2024-04-24T22:42:18.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.732+0000 7f1daea00640 1 -- 172.21.15.47:0/3572017915 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da0006d50 con 0x7f1db80cab50 2024-04-24T22:42:18.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.732+0000 7f1daea00640 1 -- 172.21.15.47:0/3572017915 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0003bb0 con 0x7f1db80cab50 2024-04-24T22:42:18.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.732+0000 7f1db7e00640 1 -- 172.21.15.47:0/3572017915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db813f320 con 0x7f1db80cab50 2024-04-24T22:42:18.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.732+0000 7f1db7e00640 1 -- 172.21.15.47:0/3572017915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db813f8e0 con 0x7f1db80cab50 2024-04-24T22:42:18.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.733+0000 7f1db7e00640 1 -- 172.21.15.47:0/3572017915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db813fff0 con 0x7f1db80cab50 2024-04-24T22:42:18.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.734+0000 7f1daea00640 1 -- 172.21.15.47:0/3572017915 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1da000a850 con 0x7f1db80cab50 2024-04-24T22:42:18.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.735+0000 7f1daea00640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1d9406be10 0x7f1d9406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:18.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.735+0000 7f1daea00640 1 -- 172.21.15.47:0/3572017915 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f1da008ab60 con 0x7f1db80cab50 2024-04-24T22:42:18.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.735+0000 7f1db6a00640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1d9406be10 0x7f1d9406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:18.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.736+0000 7f1db6a00640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1d9406be10 0x7f1d9406e2d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1da8034000 tx=0x7f1da802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:18.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:18.740+0000 7f1daea00640 1 -- 172.21.15.47:0/3572017915 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1da0020020 con 0x7f1db80cab50 2024-04-24T22:42:19.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.008+0000 7f1db7e00640 1 -- 172.21.15.47:0/3572017915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1db806d950 con 0x7f1db80cab50 2024-04-24T22:42:19.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.008+0000 7f1daea00640 1 -- 172.21.15.47:0/3572017915 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f1da0059480 con 0x7f1db80cab50 2024-04-24T22:42:19.011 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 -- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1d9406be10 msgr2=0x7f1d9406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1d9406be10 0x7f1d9406e2d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1da8034000 tx=0x7f1da802e000 comp rx=0 tx=0).stop 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 -- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80cab50 msgr2=0x7f1db8152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80cab50 0x7f1db8152bb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1da000c7c0 tx=0x7f1da000cc80 comp rx=0 tx=0).stop 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 -- 172.21.15.47:0/3572017915 shutdown_connections 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f1d9406be10 0x7f1d9406e2d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1db81581b0 0x7f1db81530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1db80cab50 0x7f1db8152bb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 --2- 172.21.15.47:0/3572017915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1db80ca770 0x7f1db80cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 -- 172.21.15.47:0/3572017915 >> 172.21.15.47:0/3572017915 conn(0x7f1db8072c40 msgr2=0x7f1db81566e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 -- 172.21.15.47:0/3572017915 shutdown_connections 2024-04-24T22:42:19.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:19.011+0000 7f1db7e00640 1 -- 172.21.15.47:0/3572017915 wait complete. 2024-04-24T22:42:19.439 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:19.439 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 readers = 5379322.00 kB/sec 2024-04-24T22:42:19.439 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 readers = 5353594.59 kB/sec 2024-04-24T22:42:19.439 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 5379322.00 kB/sec 2024-04-24T22:42:19.439 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 5379322.00 kB/sec 2024-04-24T22:42:19.440 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 5379322.00 kB/sec 2024-04-24T22:42:19.440 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 1048576.00 kB 2024-04-24T22:42:21.832 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:21.832 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 re-readers = 3376435.75 kB/sec 2024-04-24T22:42:21.833 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 re-readers = 3365405.99 kB/sec 2024-04-24T22:42:21.833 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 3376435.75 kB/sec 2024-04-24T22:42:21.833 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 3376435.75 kB/sec 2024-04-24T22:42:21.833 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 3376435.75 kB/sec 2024-04-24T22:42:21.833 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 1048576.00 kB 2024-04-24T22:42:22.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:42:22.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.250+0000 7efd31e00640 1 Processor -- start 2024-04-24T22:42:22.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.250+0000 7efd31e00640 1 -- start start 2024-04-24T22:42:22.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.251+0000 7efd31e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:22.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd31e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:22.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd31e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd2c15c3d0 con 0x7efd2c06bb90 2024-04-24T22:42:22.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd31e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7efd2c158bf0 con 0x7efd2c073040 2024-04-24T22:42:22.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd31e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efd2c15c6b0 con 0x7efd2c072c60 2024-04-24T22:42:22.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd2be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:22.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd30a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:22.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd2be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42722/0 (socket says 172.21.15.47:42722) 2024-04-24T22:42:22.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 learned_addr learned my addr 172.21.15.47:0/1659630493 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:22.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 msgr2=0x7efd2c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:42:22.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 msgr2=0x7efd2c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:42:22.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:22.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd30a00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 msgr2=0x7efd2c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:42:22.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd30a00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 msgr2=0x7efd2c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:42:22.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:22.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:22.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.252+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.453+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.453+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.454+0000 7efd30a00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 msgr2=0x7efd2c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:42:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.454+0000 7efd30a00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 msgr2=0x7efd2c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:42:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.454+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.454+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:42:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.454+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 msgr2=0x7efd2c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:42:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.454+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 msgr2=0x7efd2c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:42:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.454+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.454+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:42:22.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.855+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:22.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.855+0000 7efd30a00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 msgr2=0x7efd2c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:42:22.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.855+0000 7efd30a00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 msgr2=0x7efd2c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:42:22.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.855+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:22.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.855+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:42:22.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.856+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:22.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.856+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 msgr2=0x7efd2c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:42:22.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.856+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 msgr2=0x7efd2c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:42:22.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.856+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:22.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:22.857+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:42:23.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:23.657+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:23.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:23.657+0000 7efd30a00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 msgr2=0x7efd2c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:42:23.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:23.657+0000 7efd30a00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 msgr2=0x7efd2c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:42:23.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:23.657+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:23.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:23.657+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:42:23.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:23.658+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:23.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:23.659+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 msgr2=0x7efd2c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:42:23.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:23.659+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 msgr2=0x7efd2c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:42:23.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:23.659+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:23.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:23.659+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:42:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.252+0000 7efd2a000640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 msgr2=0x7efd2c0c2d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.252+0000 7efd2a000640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.113:3300/0 conn(0x7efd2c072c60 0x7efd2c0c2d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.252+0000 7efd2a000640 1 -- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 msgr2=0x7efd2c1411f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.252+0000 7efd2a000640 1 --2- 172.21.15.47:0/1659630493 >> v2:172.21.15.47:3300/0 conn(0x7efd2c073040 0x7efd2c1411f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.252+0000 7efd2a000640 1 -- 172.21.15.47:0/1659630493 >> v1:172.21.15.113:6789/0 conn(0x7efd2c06bb90 legacy=0x7efd2c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.252+0000 7efd2a000640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd0c0072c0 0x7efd0c009780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.252+0000 7efd2a000640 1 -- 172.21.15.47:0/1659630493 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd2c1585a0 con 0x7efd0c003690 2024-04-24T22:42:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.252+0000 7efd2a000640 1 -- 172.21.15.47:0/1659630493 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efd2c158bf0 con 0x7efd0c0072c0 2024-04-24T22:42:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.252+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd0c0072c0 msgr2=0x7efd0c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.253+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd0c0072c0 0x7efd0c009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.430+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 788077917 0 0) 0x7efd2c1585a0 con 0x7efd0c003690 2024-04-24T22:42:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.431+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd10003520 con 0x7efd0c003690 2024-04-24T22:42:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.431+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2450566631 0 0) 0x7efd10003520 con 0x7efd0c003690 2024-04-24T22:42:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.431+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd0c0072c0 msgr2=0x7efd0c009780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd2b400640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd0c0072c0 0x7efd0c009780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c00b000 con 0x7efd0c003690 2024-04-24T22:42:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 567141988 0 0) 0x7efd1c00d580 con 0x7efd0c003690 2024-04-24T22:42:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 >> v1:172.21.15.47:6789/0 conn(0x7efd0c003690 legacy=0x7efd0c005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd31400640 1 -- 172.21.15.47:0/1659630493 reap_dead start 2024-04-24T22:42:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd2b400640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 0x7efd10003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd2b400640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd2b400640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd10005370 0x7efd1000b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c158bf0 con 0x7efd10005370 2024-04-24T22:42:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd1000c2c0 con 0x7efd10004800 2024-04-24T22:42:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efd1000c4c0 con 0x7efd100039b0 2024-04-24T22:42:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd30a00640 1 -- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 msgr2=0x7efd10003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.432+0000 7efd30a00640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 0x7efd10003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.433+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.433+0000 7efd31400640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd10005370 0x7efd1000b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.434+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 msgr2=0x7efd10003db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.434+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 0x7efd10003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.434+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd10005370 msgr2=0x7efd1000b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.434+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd10005370 0x7efd1000b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.434+0000 7efd2be00640 1 -- 172.21.15.47:0/1659630493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd10003520 con 0x7efd10004800 2024-04-24T22:42:25.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.437+0000 7efd2be00640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd10004c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efd18003790 tx=0x7efd10005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e297b3a9216be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:25.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.438+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18031070 con 0x7efd10004800 2024-04-24T22:42:25.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.438+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1802b710 con 0x7efd10004800 2024-04-24T22:42:25.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.438+0000 7efd2b400640 1 -- 172.21.15.47:0/1659630493 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1800f040 con 0x7efd10004800 2024-04-24T22:42:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.439+0000 7efd31e00640 1 -- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 msgr2=0x7efd10004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.439+0000 7efd31e00640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd10004c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efd18003790 tx=0x7efd10005830 comp rx=0 tx=0).stop 2024-04-24T22:42:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.439+0000 7efd31e00640 1 -- 172.21.15.47:0/1659630493 shutdown_connections 2024-04-24T22:42:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.439+0000 7efd31e00640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd10005370 0x7efd1000b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.439+0000 7efd31e00640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd10004c80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.439+0000 7efd31e00640 1 --2- 172.21.15.47:0/1659630493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 0x7efd10003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.439+0000 7efd31e00640 1 -- 172.21.15.47:0/1659630493 >> 172.21.15.47:0/1659630493 conn(0x7efd2c0cb900 msgr2=0x7efd2c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.440+0000 7efd31e00640 1 -- 172.21.15.47:0/1659630493 shutdown_connections 2024-04-24T22:42:25.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.440+0000 7efd31e00640 1 -- 172.21.15.47:0/1659630493 wait complete. 2024-04-24T22:42:25.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.445+0000 7efd31e00640 1 Processor -- start 2024-04-24T22:42:25.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.445+0000 7efd31e00640 1 -- start start 2024-04-24T22:42:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.447+0000 7efd31e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 0x7efd2c1f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd31e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd2c1f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd31e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd10005370 0x7efd2c0bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd31e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c159080 con 0x7efd10005370 2024-04-24T22:42:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd31e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c158270 con 0x7efd10004800 2024-04-24T22:42:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd31e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efd2c159200 con 0x7efd100039b0 2024-04-24T22:42:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd30a00640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 msgr2=0x7efd2c1f09d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd30a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 0x7efd2c1f09d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd2be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd2c1f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd2be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd2c1f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:57806/0 (socket says 172.21.15.47:57806) 2024-04-24T22:42:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd2be00640 1 -- 172.21.15.47:0/785593166 learned_addr learned my addr 172.21.15.47:0/785593166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.448+0000 7efd31400640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd10005370 0x7efd2c0bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.449+0000 7efd2be00640 1 -- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 msgr2=0x7efd2c1f09d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:42:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.449+0000 7efd2be00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 0x7efd2c1f09d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.449+0000 7efd2be00640 1 -- 172.21.15.47:0/785593166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd10005370 msgr2=0x7efd2c0bdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.449+0000 7efd2be00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd10005370 0x7efd2c0bdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.449+0000 7efd2be00640 1 -- 172.21.15.47:0/785593166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c0c6f80 con 0x7efd10004800 2024-04-24T22:42:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.449+0000 7efd2be00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd2c1f1ec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd180038b0 tx=0x7efd18004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.450+0000 7efd28c00640 1 -- 172.21.15.47:0/785593166 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18044030 con 0x7efd10004800 2024-04-24T22:42:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.450+0000 7efd28c00640 1 -- 172.21.15.47:0/785593166 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd18004000 con 0x7efd10004800 2024-04-24T22:42:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.450+0000 7efd28c00640 1 -- 172.21.15.47:0/785593166 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18035a40 con 0x7efd10004800 2024-04-24T22:42:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.450+0000 7efd31e00640 1 -- 172.21.15.47:0/785593166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2c0c7240 con 0x7efd10004800 2024-04-24T22:42:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.450+0000 7efd31e00640 1 -- 172.21.15.47:0/785593166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2c0c7800 con 0x7efd10004800 2024-04-24T22:42:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.452+0000 7efd31e00640 1 -- 172.21.15.47:0/785593166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2c068d50 con 0x7efd10004800 2024-04-24T22:42:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.453+0000 7efd28c00640 1 -- 172.21.15.47:0/785593166 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd18047070 con 0x7efd10004800 2024-04-24T22:42:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.454+0000 7efd28c00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd0c06c730 0x7efd0c06ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.454+0000 7efd28c00640 1 -- 172.21.15.47:0/785593166 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7efd180abe30 con 0x7efd10004800 2024-04-24T22:42:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.454+0000 7efd30a00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd0c06c730 0x7efd0c06ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.455+0000 7efd30a00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd0c06c730 0x7efd0c06ebf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd1c006fd0 tx=0x7efd1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.459+0000 7efd28c00640 1 -- 172.21.15.47:0/785593166 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd1807a7d0 con 0x7efd10004800 2024-04-24T22:42:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.755+0000 7efd31e00640 1 -- 172.21.15.47:0/785593166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efd2c0c9b20 con 0x7efd10004800 2024-04-24T22:42:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.756+0000 7efd28c00640 1 -- 172.21.15.47:0/785593166 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7efd1807a0d0 con 0x7efd10004800 2024-04-24T22:42:25.759 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":24,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:42:25.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.759+0000 7efd31e00640 1 -- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd0c06c730 msgr2=0x7efd0c06ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:25.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.759+0000 7efd31e00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd0c06c730 0x7efd0c06ebf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd1c006fd0 tx=0x7efd1c008040 comp rx=0 tx=0).stop 2024-04-24T22:42:25.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.759+0000 7efd31e00640 1 -- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 msgr2=0x7efd2c1f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:25.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.759+0000 7efd31e00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd2c1f1ec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd180038b0 tx=0x7efd18004730 comp rx=0 tx=0).stop 2024-04-24T22:42:25.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.759+0000 7efd31e00640 1 -- 172.21.15.47:0/785593166 shutdown_connections 2024-04-24T22:42:25.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.759+0000 7efd31e00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd0c06c730 0x7efd0c06ebf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.759+0000 7efd31e00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd10005370 0x7efd2c0bdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.759+0000 7efd31e00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd10004800 0x7efd2c1f1ec0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.759+0000 7efd31e00640 1 --2- 172.21.15.47:0/785593166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd100039b0 0x7efd2c1f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.759+0000 7efd31e00640 1 -- 172.21.15.47:0/785593166 >> 172.21.15.47:0/785593166 conn(0x7efd2c0cb900 msgr2=0x7efd2c0707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.760+0000 7efd31e00640 1 -- 172.21.15.47:0/785593166 shutdown_connections 2024-04-24T22:42:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.760+0000 7efd31e00640 1 -- 172.21.15.47:0/785593166 wait complete. 2024-04-24T22:42:25.783 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-24T22:42:25.783 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-24T22:42:25.784 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:42:25.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.986+0000 7f01d9000640 1 Processor -- start 2024-04-24T22:42:25.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.986+0000 7f01d9000640 1 -- start start 2024-04-24T22:42:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.988+0000 7f01d9000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01d41514f0 0x7f01d406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.988+0000 7f01d9000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01d406e6f0 0x7f01d40c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.988+0000 7f01d9000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d4071570 con 0x7f01d40c6850 2024-04-24T22:42:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.988+0000 7f01d9000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f01d415b470 con 0x7f01d41514f0 2024-04-24T22:42:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.988+0000 7f01d9000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f01d415a8a0 con 0x7f01d406e6f0 2024-04-24T22:42:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.988+0000 7f01d3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01d41514f0 0x7f01d406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.988+0000 7f01d3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01d41514f0 0x7f01d406e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34884/0 (socket says 172.21.15.47:34884) 2024-04-24T22:42:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.988+0000 7f01d3400640 1 -- 172.21.15.47:0/2195724222 learned_addr learned my addr 172.21.15.47:0/2195724222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.989+0000 7f01d2a00640 1 --2- 172.21.15.47:0/2195724222 >> v2:172.21.15.113:3300/0 conn(0x7f01d406e6f0 0x7f01d40c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.989+0000 7f01d2a00640 1 -- 172.21.15.47:0/2195724222 >> v2:172.21.15.47:3300/0 conn(0x7f01d41514f0 msgr2=0x7f01d406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.989+0000 7f01d2a00640 1 --2- 172.21.15.47:0/2195724222 >> v2:172.21.15.47:3300/0 conn(0x7f01d41514f0 0x7f01d406e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.989+0000 7f01d2a00640 1 -- 172.21.15.47:0/2195724222 >> v1:172.21.15.113:6789/0 conn(0x7f01d40c6850 legacy=0x7f01d40707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.989+0000 7f01d2a00640 1 -- 172.21.15.47:0/2195724222 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d41518d0 con 0x7f01d406e6f0 2024-04-24T22:42:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.990+0000 7f01d2a00640 1 --2- 172.21.15.47:0/2195724222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d406e6f0 0x7f01d40c6310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f01d4154dd0 tx=0x7f01c002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ec5ca37fbddca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.990+0000 7f01d2000640 1 -- 172.21.15.47:0/2195724222 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0037070 con 0x7f01d406e6f0 2024-04-24T22:42:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.990+0000 7f01d2000640 1 -- 172.21.15.47:0/2195724222 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c0032440 con 0x7f01d406e6f0 2024-04-24T22:42:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.990+0000 7f01d2000640 1 -- 172.21.15.47:0/2195724222 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0032810 con 0x7f01d406e6f0 2024-04-24T22:42:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.991+0000 7f01d9000640 1 -- 172.21.15.47:0/2195724222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d406e6f0 msgr2=0x7f01d40c6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.991+0000 7f01d9000640 1 --2- 172.21.15.47:0/2195724222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d406e6f0 0x7f01d40c6310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f01d4154dd0 tx=0x7f01c002a1a0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.991+0000 7f01d9000640 1 -- 172.21.15.47:0/2195724222 shutdown_connections 2024-04-24T22:42:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.991+0000 7f01d9000640 1 --2- 172.21.15.47:0/2195724222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d406e6f0 0x7f01d40c6310 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.991+0000 7f01d9000640 1 --2- 172.21.15.47:0/2195724222 >> v2:172.21.15.47:3300/0 conn(0x7f01d41514f0 0x7f01d406e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.991+0000 7f01d9000640 1 -- 172.21.15.47:0/2195724222 >> 172.21.15.47:0/2195724222 conn(0x7f01d40cb900 msgr2=0x7f01d40cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.991+0000 7f01d9000640 1 -- 172.21.15.47:0/2195724222 shutdown_connections 2024-04-24T22:42:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.991+0000 7f01d9000640 1 -- 172.21.15.47:0/2195724222 wait complete. 2024-04-24T22:42:25.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.993+0000 7f01d9000640 1 Processor -- start 2024-04-24T22:42:25.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.993+0000 7f01d9000640 1 -- start start 2024-04-24T22:42:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d9000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01d406e6f0 0x7f01d41f0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d40c6850 0x7f01d41591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d41514f0 0x7f01d406b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d9000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01d415b100 con 0x7f01d40c6850 2024-04-24T22:42:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d9000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f01d415ae20 con 0x7f01d41514f0 2024-04-24T22:42:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d9000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f01d40c6c60 con 0x7f01d406e6f0 2024-04-24T22:42:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d41514f0 0x7f01d406b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d3400640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01d406e6f0 msgr2=0x7f01d41f0810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d41514f0 0x7f01d406b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:32938/0 (socket says 172.21.15.47:32938) 2024-04-24T22:42:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01d406e6f0 0x7f01d41f0810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.994+0000 7f01d3e00640 1 -- 172.21.15.47:0/3564168834 learned_addr learned my addr 172.21.15.47:0/3564168834 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.995+0000 7f01d2a00640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d40c6850 0x7f01d41591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.995+0000 7f01d3e00640 1 -- 172.21.15.47:0/3564168834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01d406e6f0 msgr2=0x7f01d41f0810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:42:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.995+0000 7f01d3e00640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01d406e6f0 0x7f01d41f0810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.995+0000 7f01d3e00640 1 -- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d40c6850 msgr2=0x7f01d41591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.995+0000 7f01d3e00640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d40c6850 0x7f01d41591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.995+0000 7f01d3e00640 1 -- 172.21.15.47:0/3564168834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d40c5200 con 0x7f01d41514f0 2024-04-24T22:42:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.995+0000 7f01d3e00640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d41514f0 0x7f01d406b160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00c3c0 tx=0x7f01cc00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.996+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc0052d0 con 0x7f01d41514f0 2024-04-24T22:42:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.996+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc005490 con 0x7f01d41514f0 2024-04-24T22:42:26.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.996+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc007260 con 0x7f01d41514f0 2024-04-24T22:42:26.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.996+0000 7f01d9000640 1 -- 172.21.15.47:0/3564168834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d4158d10 con 0x7f01d41514f0 2024-04-24T22:42:26.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.996+0000 7f01d9000640 1 -- 172.21.15.47:0/3564168834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d4159f10 con 0x7f01d41514f0 2024-04-24T22:42:26.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.997+0000 7f01d9000640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d4158860 0x7f01d406b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.997+0000 7f01d9000640 1 -- 172.21.15.47:0/3564168834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01d415a4d0 con 0x7f01d4158860 2024-04-24T22:42:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.997+0000 7f01d3400640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d4158860 0x7f01d406b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.998+0000 7f01d3400640 1 -- 172.21.15.47:0/3564168834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01d41f0d50 con 0x7f01d4158860 2024-04-24T22:42:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.998+0000 7f01d3400640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d4158860 0x7f01d406b6a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f01c40072a0 tx=0x7f01c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.998+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c400cd60 con 0x7f01d4158860 2024-04-24T22:42:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.999+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01cc007400 con 0x7f01d41514f0 2024-04-24T22:42:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:25.999+0000 7f01cb400640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f01b406bd30 0x7f01b406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.000+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f01cc08a350 con 0x7f01d41514f0 2024-04-24T22:42:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.000+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f01c4019070 con 0x7f01d4158860 2024-04-24T22:42:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.000+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d4158860 msgr2=0x7f01d406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.000+0000 7f01cb400640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d4158860 0x7f01d406b6a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f01c40072a0 tx=0x7f01c400c490 comp rx=0 tx=0).stop 2024-04-24T22:42:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.000+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d4158860 msgr2=0x7f01d406b6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.000+0000 7f01cb400640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d4158860 0x7f01d406b6a0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f01c40072a0 tx=0x7f01c400c490 comp rx=0 tx=0).stop 2024-04-24T22:42:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.000+0000 7f01d2a00640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f01b406bd30 0x7f01b406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.001+0000 7f01d2a00640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f01b406bd30 0x7f01b406e1f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f01c0034000 tx=0x7f01c002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.008+0000 7f01d9000640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d41f3fb0 0x7f01d41f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.008+0000 7f01d9000640 1 -- 172.21.15.47:0/3564168834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01d40c2750 con 0x7f01d41f3fb0 2024-04-24T22:42:26.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.008+0000 7f01d3400640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d41f3fb0 0x7f01d41f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.008+0000 7f01d3400640 1 -- 172.21.15.47:0/3564168834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f01d404f5e0 con 0x7f01d41f3fb0 2024-04-24T22:42:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.009+0000 7f01d3400640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d41f3fb0 0x7f01d41f3330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01c4018000 tx=0x7f01c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.009+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4003990 con 0x7f01d41f3fb0 2024-04-24T22:42:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.009+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5028 (secure 0 0 0) 0x7f01c40263f0 con 0x7f01d41f3fb0 2024-04-24T22:42:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.009+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d41f3fb0 msgr2=0x7f01d41f3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.009+0000 7f01cb400640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d41f3fb0 0x7f01d41f3330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01c4018000 tx=0x7f01c4004ab0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.009+0000 7f01cb400640 1 -- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d41f3fb0 msgr2=0x7f01d41f3330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:26.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.009+0000 7f01cb400640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d41f3fb0 0x7f01d41f3330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.013 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:42:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:42:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 24, 2024-04-24T22:42:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 86047, 2024-04-24T22:42:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:42:26.013 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: 1 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:26.014 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:26.015 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:42:26.016 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:26.017 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:42:26.019 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:42:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:42:26.020 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:26.024 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.025 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.026 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:42:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:42:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:26.027 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:26.028 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:26.028 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.028 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.028 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:42:26.028 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:42:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.010+0000 7f01d9000640 1 -- 172.21.15.47:0/3564168834 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f01b406bd30 msgr2=0x7f01b406e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.010+0000 7f01d9000640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f01b406bd30 0x7f01b406e1f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f01c0034000 tx=0x7f01c002e000 comp rx=0 tx=0).stop 2024-04-24T22:42:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.010+0000 7f01d9000640 1 -- 172.21.15.47:0/3564168834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d41514f0 msgr2=0x7f01d406b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.010+0000 7f01d9000640 1 --2- 172.21.15.47:0/3564168834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d41514f0 0x7f01d406b160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00c3c0 tx=0x7f01cc00c880 comp rx=0 tx=0).stop 2024-04-24T22:42:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.010+0000 7f01d3e00640 1 -- 172.21.15.47:0/3564168834 reap_dead start 2024-04-24T22:42:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.010+0000 7f01d3e00640 1 -- 172.21.15.47:0/3564168834 reap_dead start 2024-04-24T22:42:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.010+0000 7f01d9000640 1 -- 172.21.15.47:0/3564168834 shutdown_connections 2024-04-24T22:42:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.010+0000 7f01d9000640 1 -- 172.21.15.47:0/3564168834 >> 172.21.15.47:0/3564168834 conn(0x7f01d40cb900 msgr2=0x7f01d41507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.010+0000 7f01d9000640 1 -- 172.21.15.47:0/3564168834 shutdown_connections 2024-04-24T22:42:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.010+0000 7f01d9000640 1 -- 172.21.15.47:0/3564168834 wait complete. 2024-04-24T22:42:26.031 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-24T22:42:26.031 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-24T22:42:26.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:42:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.215+0000 7fc75e600640 1 Processor -- start 2024-04-24T22:42:26.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.215+0000 7fc75e600640 1 -- start start 2024-04-24T22:42:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.216+0000 7fc75e600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc758154660 0x7fc75806ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.216+0000 7fc75e600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7580c6dc0 con 0x7fc75806f120 2024-04-24T22:42:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.216+0000 7fc75e600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7580c70a0 con 0x7fc75815e5c0 2024-04-24T22:42:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.216+0000 7fc75e600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc75815da20 con 0x7fc758154660 2024-04-24T22:42:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.217+0000 7fc757e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc75806f120 0x7fc75815deb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:49314/0 (socket says 172.21.15.47:49314) 2024-04-24T22:42:26.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.217+0000 7fc757e00640 1 -- 172.21.15.47:0/3732004720 learned_addr learned my addr 172.21.15.47:0/3732004720 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:42:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.217+0000 7fc75d200640 1 -- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc758154660 msgr2=0x7fc75806ebe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.217+0000 7fc75d200640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc758154660 0x7fc75806ebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.217+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 966290915 0 0) 0x7fc7580c6dc0 con 0x7fc75806f120 2024-04-24T22:42:26.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.217+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc740003520 con 0x7fc75806f120 2024-04-24T22:42:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1390173218 0 0) 0x7fc7580c70a0 con 0x7fc75815e5c0 2024-04-24T22:42:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7580c6dc0 con 0x7fc75815e5c0 2024-04-24T22:42:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2782823598 0 0) 0x7fc740003520 con 0x7fc75806f120 2024-04-24T22:42:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc758154660 msgr2=0x7fc75806ebe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc758154660 0x7fc75806ebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 >> v1:172.21.15.113:6789/0 conn(0x7fc75815e5c0 legacy=0x7fc7580c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7580c7380 con 0x7fc75806f120 2024-04-24T22:42:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 567141988 0 0) 0x7fc7440015a0 con 0x7fc75806f120 2024-04-24T22:42:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 >> v1:172.21.15.47:6789/0 conn(0x7fc75806f120 legacy=0x7fc75815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc740003b70 0x7fc740003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc758154660 0x7fc740004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740004e60 0x7fc74000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc75815da20 con 0x7fc740003b70 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc740005560 con 0x7fc740004e60 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc74000bf00 con 0x7fc758154660 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757e00640 1 -- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc758154660 msgr2=0x7fc740004800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc757e00640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc758154660 0x7fc740004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc75d200640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc740003b70 0x7fc740003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.218+0000 7fc75dc00640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740004e60 0x7fc74000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.219+0000 7fc75d200640 1 -- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc758154660 msgr2=0x7fc740004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:26.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.219+0000 7fc75d200640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc758154660 0x7fc740004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.219+0000 7fc75d200640 1 -- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740004e60 msgr2=0x7fc74000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:26.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.219+0000 7fc75d200640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740004e60 0x7fc74000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.219+0000 7fc75d200640 1 -- 172.21.15.47:0/3732004720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc740003520 con 0x7fc740003b70 2024-04-24T22:42:26.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.219+0000 7fc75dc00640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740004e60 0x7fc74000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:26.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.220+0000 7fc75d200640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc740003b70 0x7fc740003f90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009900 tx=0x7fc740005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6b9d2ac889d4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.220+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c009500 con 0x7fc740003b70 2024-04-24T22:42:26.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.220+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74c02e070 con 0x7fc740003b70 2024-04-24T22:42:26.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.220+0000 7fc757400640 1 -- 172.21.15.47:0/3732004720 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c003df0 con 0x7fc740003b70 2024-04-24T22:42:26.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.220+0000 7fc75e600640 1 -- 172.21.15.47:0/3732004720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc740003b70 msgr2=0x7fc740003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.220+0000 7fc75e600640 1 --2- 172.21.15.47:0/3732004720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc740003b70 0x7fc740003f90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009900 tx=0x7fc740005830 comp rx=0 tx=0).stop 2024-04-24T22:42:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.220+0000 7fc75dc00640 1 -- 172.21.15.47:0/3732004720 reap_dead start 2024-04-24T22:42:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.220+0000 7fc75e600640 1 -- 172.21.15.47:0/3732004720 shutdown_connections 2024-04-24T22:42:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.220+0000 7fc75e600640 1 -- 172.21.15.47:0/3732004720 >> 172.21.15.47:0/3732004720 conn(0x7fc75806cee0 msgr2=0x7fc75806d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.221+0000 7fc75e600640 1 -- 172.21.15.47:0/3732004720 shutdown_connections 2024-04-24T22:42:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.221+0000 7fc75e600640 1 -- 172.21.15.47:0/3732004720 wait complete. 2024-04-24T22:42:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.222+0000 7fc75e600640 1 Processor -- start 2024-04-24T22:42:26.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.222+0000 7fc75e600640 1 -- start start 2024-04-24T22:42:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.223+0000 7fc75e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740003b70 0x7fc75815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.223+0000 7fc75e600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7580cafc0 0x7fc7580cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.223+0000 7fc75e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75815ba40 0x7fc75815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.223+0000 7fc75e600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc758154a40 con 0x7fc75815ba40 2024-04-24T22:42:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.223+0000 7fc75e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc7580c7940 con 0x7fc740003b70 2024-04-24T22:42:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.223+0000 7fc75e600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc758071260 con 0x7fc7580cafc0 2024-04-24T22:42:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc757e00640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7580cafc0 msgr2=0x7fc7580cb4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3301/0 2024-04-24T22:42:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc757e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7580cafc0 0x7fc7580cb4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:26.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc75d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740003b70 0x7fc75815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc75dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75815ba40 0x7fc75815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc75d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740003b70 0x7fc75815d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:32968/0 (socket says 172.21.15.47:32968) 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc75d200640 1 -- 172.21.15.47:0/3483580970 learned_addr learned my addr 172.21.15.47:0/3483580970 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc75dc00640 1 -- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7580cafc0 msgr2=0x7fc7580cb4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc75dc00640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7580cafc0 0x7fc7580cb4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc75dc00640 1 -- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740003b70 msgr2=0x7fc75815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc75dc00640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740003b70 0x7fc75815d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc75dc00640 1 -- 172.21.15.47:0/3483580970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75815d810 con 0x7fc75815ba40 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.224+0000 7fc75d200640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc740003b70 0x7fc75815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.225+0000 7fc75dc00640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75815ba40 0x7fc75815a710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc740003b40 tx=0x7fc75000ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.225+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750001bc0 con 0x7fc75815ba40 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.225+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc750018070 con 0x7fc75815ba40 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.225+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750003dc0 con 0x7fc75815ba40 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.225+0000 7fc75e600640 1 -- 172.21.15.47:0/3483580970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7580cba30 con 0x7fc75815ba40 2024-04-24T22:42:26.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.225+0000 7fc75e600640 1 -- 172.21.15.47:0/3483580970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc758141af0 con 0x7fc75815ba40 2024-04-24T22:42:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.226+0000 7fc75e600640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc758141dd0 0x7fc7581421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.226+0000 7fc75e600640 1 -- 172.21.15.47:0/3483580970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc7580c5bf0 con 0x7fc758141dd0 2024-04-24T22:42:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.226+0000 7fc75d200640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc758141dd0 0x7fc7581421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.227+0000 7fc75d200640 1 -- 172.21.15.47:0/3483580970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc758073b90 con 0x7fc758141dd0 2024-04-24T22:42:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.227+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc75000d860 con 0x7fc75815ba40 2024-04-24T22:42:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.227+0000 7fc75d200640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc758141dd0 0x7fc7581421b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0090b0 tx=0x7fc74c005a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.228+0000 7fc754c00640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc73806be10 0x7fc73806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.228+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc75008ad70 con 0x7fc75815ba40 2024-04-24T22:42:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.228+0000 7fc757e00640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc73806be10 0x7fc73806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.229+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c005480 con 0x7fc758141dd0 2024-04-24T22:42:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.229+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc74c02e040 con 0x7fc758141dd0 2024-04-24T22:42:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.229+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc758141dd0 msgr2=0x7fc7581421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.229+0000 7fc754c00640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc758141dd0 0x7fc7581421b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0090b0 tx=0x7fc74c005a10 comp rx=0 tx=0).stop 2024-04-24T22:42:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.229+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc758141dd0 msgr2=0x7fc7581421b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.229+0000 7fc754c00640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc758141dd0 0x7fc7581421b0 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0090b0 tx=0x7fc74c005a10 comp rx=0 tx=0).stop 2024-04-24T22:42:26.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.229+0000 7fc757e00640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc73806be10 0x7fc73806e2d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc744001330 tx=0x7fc744001460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.236+0000 7fc75e600640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc75813f9b0 0x7fc75813fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:26.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.236+0000 7fc75e600640 1 -- 172.21.15.47:0/3483580970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc758157e50 con 0x7fc75813f9b0 2024-04-24T22:42:26.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.237+0000 7fc75d200640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc75813f9b0 0x7fc75813fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:26.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.237+0000 7fc75d200640 1 -- 172.21.15.47:0/3483580970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc75804f5e0 con 0x7fc75813f9b0 2024-04-24T22:42:26.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.237+0000 7fc75d200640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc75813f9b0 0x7fc75813fd90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009080 tx=0x7fc74c0090e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:26.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c046cf0 con 0x7fc75813f9b0 2024-04-24T22:42:26.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5054 (secure 0 0 0) 0x7fc74c046cf0 con 0x7fc75813f9b0 2024-04-24T22:42:26.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc75813f9b0 msgr2=0x7fc75813fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc754c00640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc75813f9b0 0x7fc75813fd90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009080 tx=0x7fc74c0090e0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc754c00640 1 -- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc75813f9b0 msgr2=0x7fc75813fd90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:26.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc754c00640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc75813f9b0 0x7fc75813fd90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:26.242 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:42:26.242 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:26.243 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:26.243 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:42:26.243 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 24, 2024-04-24T22:42:26.243 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 187806, 2024-04-24T22:42:26.243 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:42:26.243 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:42:26.244 INFO:teuthology.orchestra.run.smithi047.stdout: 1 2024-04-24T22:42:26.244 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.244 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-24T22:42:26.244 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:26.244 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:42:26.244 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:42:26.244 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:26.245 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:26.245 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:26.245 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:26.245 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:26.245 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:26.245 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:26.246 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:26.246 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:26.246 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:26.246 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:26.246 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.246 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:42:26.246 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:42:26.247 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:26.247 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:26.247 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:26.247 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:26.247 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:26.247 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:26.248 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:26.248 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:26.248 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:26.248 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:26.248 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:26.248 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.248 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:42:26.249 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:42:26.249 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:42:26.249 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:42:26.249 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:42:26.249 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:42:26.249 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:26.249 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:26.250 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:42:26.250 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:42:26.250 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:42:26.250 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:42:26.250 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:42:26.250 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:42:26.251 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:42:26.251 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:26.251 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:42:26.251 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:26.251 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:26.251 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:26.252 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:26.252 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:26.252 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:26.252 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:26.252 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:26.252 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:26.252 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:26.253 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:26.253 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.253 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:42:26.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.254 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:42:26.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.254 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:26.254 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:26.254 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:26.255 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:26.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:26.255 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:42:26.255 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:26.256 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:42:26.256 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.256 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.256 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.257 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:42:26.257 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:42:26.257 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:26.257 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:26.257 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:26.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.258 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:26.258 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:26.258 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:26.258 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:26.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:26.259 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:42:26.259 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:26.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:42:26.260 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.260 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.260 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:42:26.260 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:42:26.261 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:26.261 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:26.261 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:26.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.261 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:42:26.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:42:26.262 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:26.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:26.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:26.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:42:26.263 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:26.263 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:42:26.263 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:26.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.264 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.264 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:42:26.264 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:42:26.264 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:26.265 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:26.265 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:26.265 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.265 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.265 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:42:26.266 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:42:26.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.266 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:26.266 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:26.266 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:26.266 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.267 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.267 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:42:26.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.267 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:26.267 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:26.267 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:42:26.268 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.268 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.268 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:42:26.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.268 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:26.268 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:26.268 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7 2024-04-24T22:42:26.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.269 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.269 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:42:26.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.269 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:26.269 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:26.270 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:42:26.270 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.270 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:26.270 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:42:26.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:26.270 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:26.271 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:26.271 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-24T22:42:26.271 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:26.271 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:26.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:26.271 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:42:26.271 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:42:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc75e600640 1 -- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc73806be10 msgr2=0x7fc73806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc75e600640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc73806be10 0x7fc73806e2d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc744001330 tx=0x7fc744001460 comp rx=0 tx=0).stop 2024-04-24T22:42:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc75e600640 1 -- 172.21.15.47:0/3483580970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75815ba40 msgr2=0x7fc75815a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc75e600640 1 --2- 172.21.15.47:0/3483580970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc75815ba40 0x7fc75815a710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc740003b40 tx=0x7fc75000ff80 comp rx=0 tx=0).stop 2024-04-24T22:42:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc75dc00640 1 -- 172.21.15.47:0/3483580970 reap_dead start 2024-04-24T22:42:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc75dc00640 1 -- 172.21.15.47:0/3483580970 reap_dead start 2024-04-24T22:42:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc75e600640 1 -- 172.21.15.47:0/3483580970 shutdown_connections 2024-04-24T22:42:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.238+0000 7fc75e600640 1 -- 172.21.15.47:0/3483580970 >> 172.21.15.47:0/3483580970 conn(0x7fc75806cee0 msgr2=0x7fc758142c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.239+0000 7fc75e600640 1 -- 172.21.15.47:0/3483580970 shutdown_connections 2024-04-24T22:42:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:26.239+0000 7fc75e600640 1 -- 172.21.15.47:0/3483580970 wait complete. 2024-04-24T22:42:26.275 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-24T22:42:26.275 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-24T22:42:28.293 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:28.294 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:28.294 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:28.294 INFO:tasks.workunit.client.0.smithi047.stdout:iozone test complete. 2024-04-24T22:42:28.297 INFO:tasks.workunit.client.0.smithi047.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-24T22:42:28.392 INFO:tasks.workunit.client.0.smithi047.stdout: Iozone: Performance Test of File I/O 2024-04-24T22:42:28.392 INFO:tasks.workunit.client.0.smithi047.stdout: Version $Revision: 3.494 $ 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: Compiled for 64 bit mode. 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: Build: linux-AMD64 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-24T22:42:28.393 INFO:tasks.workunit.client.0.smithi047.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-24T22:42:28.394 INFO:tasks.workunit.client.0.smithi047.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-24T22:42:28.394 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:28.394 INFO:tasks.workunit.client.0.smithi047.stdout: Run began: Wed Apr 24 22:42:28 2024 2024-04-24T22:42:28.394 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:42:28.394 INFO:tasks.workunit.client.0.smithi047.stdout: Include close in write timing 2024-04-24T22:42:28.394 INFO:tasks.workunit.client.0.smithi047.stdout: Include fsync in write timing 2024-04-24T22:42:28.394 INFO:tasks.workunit.client.0.smithi047.stdout: File size set to 10485760 kB 2024-04-24T22:42:28.394 INFO:tasks.workunit.client.0.smithi047.stdout: Record Size 1024 kB 2024-04-24T22:42:28.394 INFO:tasks.workunit.client.0.smithi047.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-24T22:42:28.395 INFO:tasks.workunit.client.0.smithi047.stdout: Output is in kBytes/sec 2024-04-24T22:42:28.395 INFO:tasks.workunit.client.0.smithi047.stdout: Time Resolution = 0.000001 seconds. 2024-04-24T22:42:28.395 INFO:tasks.workunit.client.0.smithi047.stdout: Processor cache size set to 1024 kBytes. 2024-04-24T22:42:28.395 INFO:tasks.workunit.client.0.smithi047.stdout: Processor cache line size set to 32 bytes. 2024-04-24T22:42:28.395 INFO:tasks.workunit.client.0.smithi047.stdout: File stride size set to 17 * record size. 2024-04-24T22:42:28.395 INFO:tasks.workunit.client.0.smithi047.stdout: Throughput test with 1 process 2024-04-24T22:42:28.396 INFO:tasks.workunit.client.0.smithi047.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-04-24T22:42:40.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:42:40.711 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:42:46.276 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-24T22:42:46.276 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-24T22:42:46.276 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T22:42:46.277 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-24T22:42:46.279 INFO:tasks.ceph.mon.c:Started 2024-04-24T22:42:46.280 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-24T22:42:46.280 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:42:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.561+0000 7f7841600640 1 Processor -- start 2024-04-24T22:42:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.562+0000 7f7841600640 1 -- start start 2024-04-24T22:42:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.564+0000 7f7841600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f783c070890 0x7f783c0c1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.564+0000 7f7841600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c0c7250 0x7f783c0c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.564+0000 7f7841600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f783c156ce0 con 0x7f783c150a40 2024-04-24T22:42:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.564+0000 7f7841600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f783c0c79f0 con 0x7f783c070890 2024-04-24T22:42:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.564+0000 7f7841600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f783c0c7d90 con 0x7f783c0c7250 2024-04-24T22:42:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.566+0000 7f7833e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f783c070890 0x7f783c0c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:46.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.566+0000 7f7833e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f783c070890 0x7f783c0c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52592/0 (socket says 172.21.15.47:52592) 2024-04-24T22:42:46.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.566+0000 7f7833e00640 1 -- 172.21.15.47:0/1685920540 learned_addr learned my addr 172.21.15.47:0/1685920540 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:46.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.567+0000 7f7833e00640 1 -- 172.21.15.47:0/1685920540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c0c7250 msgr2=0x7f783c0c0e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:46.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.567+0000 7f7833e00640 1 --2- 172.21.15.47:0/1685920540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c0c7250 0x7f783c0c0e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:46.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.567+0000 7f7833e00640 1 -- 172.21.15.47:0/1685920540 >> v1:172.21.15.47:6789/0 conn(0x7f783c150a40 legacy=0x7f783c1565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:46.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.567+0000 7f7833e00640 1 -- 172.21.15.47:0/1685920540 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c0c7fe0 con 0x7f783c070890 2024-04-24T22:42:46.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.568+0000 7f7833e00640 1 --2- 172.21.15.47:0/1685920540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c070890 0x7f783c0c1940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f782c009870 tx=0x7f782c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea45712340fb5f8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.568+0000 7f783a000640 1 -- 172.21.15.47:0/1685920540 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c037070 con 0x7f783c070890 2024-04-24T22:42:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.568+0000 7f783a000640 1 -- 172.21.15.47:0/1685920540 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f782c032440 con 0x7f783c070890 2024-04-24T22:42:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.568+0000 7f783a000640 1 -- 172.21.15.47:0/1685920540 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c037070 con 0x7f783c070890 2024-04-24T22:42:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.571+0000 7f7841600640 1 -- 172.21.15.47:0/1685920540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c070890 msgr2=0x7f783c0c1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.571+0000 7f7841600640 1 --2- 172.21.15.47:0/1685920540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c070890 0x7f783c0c1940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f782c009870 tx=0x7f782c02a2c0 comp rx=0 tx=0).stop 2024-04-24T22:42:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.571+0000 7f7841600640 1 -- 172.21.15.47:0/1685920540 shutdown_connections 2024-04-24T22:42:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.571+0000 7f7841600640 1 --2- 172.21.15.47:0/1685920540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c0c7250 0x7f783c0c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.572+0000 7f7841600640 1 --2- 172.21.15.47:0/1685920540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c070890 0x7f783c0c1940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.572+0000 7f7841600640 1 -- 172.21.15.47:0/1685920540 >> 172.21.15.47:0/1685920540 conn(0x7f783c06e580 msgr2=0x7f783c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.572+0000 7f7841600640 1 -- 172.21.15.47:0/1685920540 shutdown_connections 2024-04-24T22:42:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.572+0000 7f7841600640 1 -- 172.21.15.47:0/1685920540 wait complete. 2024-04-24T22:42:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.576+0000 7f7841600640 1 Processor -- start 2024-04-24T22:42:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.577+0000 7f7841600640 1 -- start start 2024-04-24T22:42:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.579+0000 7f7841600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c070890 0x7f783c1eb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.579+0000 7f7841600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c0c7250 0x7f783c1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.579+0000 7f7841600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f783c150a40 0x7f783c1496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.580+0000 7f783aa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c070890 0x7f783c1eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.580+0000 7f783aa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c070890 0x7f783c1eb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:55170/0 (socket says 172.21.15.47:55170) 2024-04-24T22:42:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.580+0000 7f783aa00640 1 -- 172.21.15.47:0/2715622269 learned_addr learned my addr 172.21.15.47:0/2715622269 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.580+0000 7f783b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f783c150a40 0x7f783c1496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.580+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f783c0c25f0 con 0x7f783c0c7250 2024-04-24T22:42:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.580+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f783c0c2310 con 0x7f783c150a40 2024-04-24T22:42:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.580+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f783c0c2b10 con 0x7f783c070890 2024-04-24T22:42:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.581+0000 7f783aa00640 1 -- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c070890 msgr2=0x7f783c1eb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-24T22:42:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.581+0000 7f783aa00640 1 -- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c070890 msgr2=0x7f783c1eb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:42:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.581+0000 7f783aa00640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c070890 0x7f783c1eb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.581+0000 7f783aa00640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c070890 0x7f783c1eb480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.581+0000 7f783b400640 1 -- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c070890 msgr2=0x7f783c1eb480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:42:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.582+0000 7f783b400640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c070890 0x7f783c1eb480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.582+0000 7f783b400640 1 -- 172.21.15.47:0/2715622269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c0c7250 msgr2=0x7f783c1491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.582+0000 7f783b400640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c0c7250 0x7f783c1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.582+0000 7f783b400640 1 -- 172.21.15.47:0/2715622269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c14cfb0 con 0x7f783c150a40 2024-04-24T22:42:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.582+0000 7f783b400640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f783c150a40 0x7f783c1496e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f783400be30 tx=0x7f783400bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.583+0000 7f7832a00640 1 -- 172.21.15.47:0/2715622269 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834019070 con 0x7f783c150a40 2024-04-24T22:42:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.583+0000 7f7832a00640 1 -- 172.21.15.47:0/2715622269 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7834014440 con 0x7f783c150a40 2024-04-24T22:42:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.583+0000 7f7832a00640 1 -- 172.21.15.47:0/2715622269 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834014830 con 0x7f783c150a40 2024-04-24T22:42:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.583+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783c1f8810 con 0x7f783c150a40 2024-04-24T22:42:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.584+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783c1f8ab0 con 0x7f783c150a40 2024-04-24T22:42:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.585+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783c073b90 con 0x7f783c150a40 2024-04-24T22:42:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.588+0000 7f7832a00640 1 -- 172.21.15.47:0/2715622269 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f783401c360 con 0x7f783c150a40 2024-04-24T22:42:46.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.589+0000 7f7832a00640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f781806bf30 0x7f781806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:46.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.589+0000 7f7832a00640 1 -- 172.21.15.47:0/2715622269 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f783408ae30 con 0x7f783c150a40 2024-04-24T22:42:46.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.590+0000 7f783aa00640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f781806bf30 0x7f781806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:46.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.590+0000 7f783aa00640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f781806bf30 0x7f781806e3f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7824006e90 tx=0x7f7824008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:46.593+0000 7f7832a00640 1 -- 172.21.15.47:0/2715622269 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f783c14cfb0 con 0x7f783c150a40 2024-04-24T22:42:47.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:47.148+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f783c0b14d0 con 0x7f783c150a40 2024-04-24T22:42:51.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.982+0000 7f7832a00640 1 -- 172.21.15.47:0/2715622269 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f78340596b0 con 0x7f783c150a40 2024-04-24T22:42:51.991 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":28,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:42:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.995+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f781806bf30 msgr2=0x7f781806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.995+0000 7f7841600640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f781806bf30 0x7f781806e3f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7824006e90 tx=0x7f7824008040 comp rx=0 tx=0).stop 2024-04-24T22:42:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.995+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f783c150a40 msgr2=0x7f783c1496e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.995+0000 7f7841600640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f783c150a40 0x7f783c1496e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f783400be30 tx=0x7f783400bf20 comp rx=0 tx=0).stop 2024-04-24T22:42:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.996+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 shutdown_connections 2024-04-24T22:42:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.996+0000 7f7841600640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f781806bf30 0x7f781806e3f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.996+0000 7f7841600640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f783c150a40 0x7f783c1496e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.996+0000 7f7841600640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f783c0c7250 0x7f783c1491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.996+0000 7f7841600640 1 --2- 172.21.15.47:0/2715622269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f783c070890 0x7f783c1eb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.996+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 >> 172.21.15.47:0/2715622269 conn(0x7f783c06e580 msgr2=0x7f783c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.996+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 shutdown_connections 2024-04-24T22:42:51.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:51.996+0000 7f7841600640 1 -- 172.21.15.47:0/2715622269 wait complete. 2024-04-24T22:42:55.025 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:42:55.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.264+0000 7f01b1600640 1 Processor -- start 2024-04-24T22:42:55.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.264+0000 7f01b1600640 1 -- start start 2024-04-24T22:42:55.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.265+0000 7f01b1600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01ac152960 0x7f01ac0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:55.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.265+0000 7f01b1600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:55.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01b1600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac0c9400 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01b1600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac158f20 con 0x7f01ac152960 2024-04-24T22:42:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01b1600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac1593b0 con 0x7f01ac072b80 2024-04-24T22:42:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01aa000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f01ac0c8fa0 0x7f01ac15ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:44108/0 (socket says 172.21.15.47:44108) 2024-04-24T22:42:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01aa000640 1 -- 172.21.15.47:0/2592853989 learned_addr learned my addr 172.21.15.47:0/2592853989 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:42:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01ab400640 1 --2- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01aaa00640 1 --2- 172.21.15.47:0/2592853989 >> v2:172.21.15.113:3300/0 conn(0x7f01ac152960 0x7f01ac0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01ab400640 1 -- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 msgr2=0x7f01ac072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T22:42:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01ab400640 1 -- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 msgr2=0x7f01ac072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:42:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01ab400640 1 --2- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:55.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.266+0000 7f01ab400640 1 --2- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac072ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:55.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.267+0000 7f01aaa00640 1 -- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 msgr2=0x7f01ac072ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:55.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.267+0000 7f01a9600640 1 -- 172.21.15.47:0/2592853989 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3380043953 0 0) 0x7f01ac0c9400 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.267+0000 7f01aaa00640 1 --2- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac072ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:55.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.267+0000 7f01aaa00640 1 -- 172.21.15.47:0/2592853989 >> v1:172.21.15.113:6789/0 conn(0x7f01ac0c8fa0 legacy=0x7f01ac15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:55.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.267+0000 7f01aaa00640 1 -- 172.21.15.47:0/2592853989 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac0c7e50 con 0x7f01ac152960 2024-04-24T22:42:55.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.269+0000 7f01aaa00640 1 --2- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01ac152960 0x7f01ac0c7750 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f019c0097a0 tx=0x7f019c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7492c14acb8ea64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:55.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.270+0000 7f01a9600640 1 -- 172.21.15.47:0/2592853989 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c037070 con 0x7f01ac152960 2024-04-24T22:42:55.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.270+0000 7f01a9600640 1 -- 172.21.15.47:0/2592853989 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019c032440 con 0x7f01ac152960 2024-04-24T22:42:55.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.270+0000 7f01a9600640 1 -- 172.21.15.47:0/2592853989 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c032810 con 0x7f01ac152960 2024-04-24T22:42:55.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.271+0000 7f01b1600640 1 -- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01ac152960 msgr2=0x7f01ac0c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:55.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.271+0000 7f01b1600640 1 --2- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01ac152960 0x7f01ac0c7750 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f019c0097a0 tx=0x7f019c02a210 comp rx=0 tx=0).stop 2024-04-24T22:42:55.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.271+0000 7f01b1600640 1 -- 172.21.15.47:0/2592853989 shutdown_connections 2024-04-24T22:42:55.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.271+0000 7f01b1600640 1 --2- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:55.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.271+0000 7f01b1600640 1 --2- 172.21.15.47:0/2592853989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01ac152960 0x7f01ac0c7750 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:55.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.271+0000 7f01b1600640 1 -- 172.21.15.47:0/2592853989 >> 172.21.15.47:0/2592853989 conn(0x7f01ac06e580 msgr2=0x7f01ac06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:55.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.271+0000 7f01b1600640 1 -- 172.21.15.47:0/2592853989 shutdown_connections 2024-04-24T22:42:55.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.271+0000 7f01b1600640 1 -- 172.21.15.47:0/2592853989 wait complete. 2024-04-24T22:42:55.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.273+0000 7f01b1600640 1 Processor -- start 2024-04-24T22:42:55.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.273+0000 7f01b1600640 1 -- start start 2024-04-24T22:42:55.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.275+0000 7f01b1600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac0c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:55.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.275+0000 7f01b1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac0c8fa0 0x7f01ac144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:55.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.275+0000 7f01b1600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01ac152960 0x7f01ac15b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:55.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.275+0000 7f01b1600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1588d0 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.275+0000 7f01b1600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac0c8130 con 0x7f01ac152960 2024-04-24T22:42:55.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.275+0000 7f01b1600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac158bb0 con 0x7f01ac072b80 2024-04-24T22:42:55.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.275+0000 7f01aaa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac0c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:55.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.275+0000 7f01aaa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac0c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:55188/0 (socket says 172.21.15.47:55188) 2024-04-24T22:42:55.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.275+0000 7f01aaa00640 1 -- 172.21.15.47:0/3068630242 learned_addr learned my addr 172.21.15.47:0/3068630242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:55.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.276+0000 7f01aaa00640 1 -- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 msgr2=0x7f01ac0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-24T22:42:55.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.276+0000 7f01aaa00640 1 -- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 msgr2=0x7f01ac0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.276+0000 7f01aaa00640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac0c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.276+0000 7f01aaa00640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac0c3410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.276+0000 7f01aa000640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac0c8fa0 0x7f01ac144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.276+0000 7f01ab400640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01ac152960 0x7f01ac15b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.277+0000 7f01aa000640 1 -- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 msgr2=0x7f01ac0c3410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.277+0000 7f01aa000640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac0c3410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.277+0000 7f01aa000640 1 -- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01ac152960 msgr2=0x7f01ac15b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.277+0000 7f01aa000640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01ac152960 0x7f01ac15b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.277+0000 7f01aa000640 1 -- 172.21.15.47:0/3068630242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac0c5340 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.277+0000 7f01aa000640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac0c8fa0 0x7f01ac144a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f019800c450 tx=0x7f019800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.278+0000 7f01a2a00640 1 -- 172.21.15.47:0/3068630242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198018070 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.278+0000 7f01b1600640 1 -- 172.21.15.47:0/3068630242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac072500 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.278+0000 7f01b1600640 1 -- 172.21.15.47:0/3068630242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac0c55e0 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.278+0000 7f01a2a00640 1 -- 172.21.15.47:0/3068630242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01980074e0 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.278+0000 7f01a2a00640 1 -- 172.21.15.47:0/3068630242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198004a20 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.280+0000 7f01b1600640 1 -- 172.21.15.47:0/3068630242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0184005290 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.281+0000 7f01a2a00640 1 -- 172.21.15.47:0/3068630242 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0198004bc0 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.283+0000 7f01a2a00640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f018c06be30 0x7f018c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.283+0000 7f01a2a00640 1 -- 172.21.15.47:0/3068630242 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f019808a1c0 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.284+0000 7f01aaa00640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f018c06be30 0x7f018c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.285+0000 7f01aaa00640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f018c06be30 0x7f018c06e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f019c035040 tx=0x7f019c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:55.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.288+0000 7f01a2a00640 1 -- 172.21.15.47:0/3068630242 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01980589d0 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.810+0000 7f01b1600640 1 -- 172.21.15.47:0/3068630242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0184005580 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.816+0000 7f01a2a00640 1 -- 172.21.15.47:0/3068630242 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f01980582d0 con 0x7f01ac0c8fa0 2024-04-24T22:42:55.821 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:42:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.824+0000 7f01b1600640 1 -- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f018c06be30 msgr2=0x7f018c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.824+0000 7f01b1600640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f018c06be30 0x7f018c06e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f019c035040 tx=0x7f019c02e000 comp rx=0 tx=0).stop 2024-04-24T22:42:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.824+0000 7f01b1600640 1 -- 172.21.15.47:0/3068630242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac0c8fa0 msgr2=0x7f01ac144a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.824+0000 7f01b1600640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac0c8fa0 0x7f01ac144a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f019800c450 tx=0x7f019800c910 comp rx=0 tx=0).stop 2024-04-24T22:42:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.825+0000 7f01b1600640 1 -- 172.21.15.47:0/3068630242 shutdown_connections 2024-04-24T22:42:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.825+0000 7f01b1600640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f018c06be30 0x7f018c06e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.825+0000 7f01b1600640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01ac152960 0x7f01ac15b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.825+0000 7f01b1600640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01ac0c8fa0 0x7f01ac144a80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.825+0000 7f01b1600640 1 --2- 172.21.15.47:0/3068630242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01ac072b80 0x7f01ac0c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.825+0000 7f01b1600640 1 -- 172.21.15.47:0/3068630242 >> 172.21.15.47:0/3068630242 conn(0x7f01ac06e580 msgr2=0x7f01ac0c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.825+0000 7f01b1600640 1 -- 172.21.15.47:0/3068630242 shutdown_connections 2024-04-24T22:42:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:55.825+0000 7f01b1600640 1 -- 172.21.15.47:0/3068630242 wait complete. 2024-04-24T22:42:55.849 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-24T22:42:55.849 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-24T22:42:55.849 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:42:56.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.165+0000 7fb2c0a00640 1 Processor -- start 2024-04-24T22:42:56.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.165+0000 7fb2c0a00640 1 -- start start 2024-04-24T22:42:56.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.167+0000 7fb2c0a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2bc151780 0x7fb2bc0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.167+0000 7fb2c0a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2bc0c1a00 0x7fb2bc0c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.167+0000 7fb2c0a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2bc15aa30 con 0x7fb2bc072f60 2024-04-24T22:42:56.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.167+0000 7fb2c0a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb2bc15ad10 con 0x7fb2bc151780 2024-04-24T22:42:56.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.167+0000 7fb2c0a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb2bc15aff0 con 0x7fb2bc0c1a00 2024-04-24T22:42:56.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.168+0000 7fb2b9e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2bc151780 0x7fb2bc0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.168+0000 7fb2b9400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2bc0c1a00 0x7fb2bc0c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.168+0000 7fb2b9e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2bc151780 0x7fb2bc0c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40778/0 (socket says 172.21.15.47:40778) 2024-04-24T22:42:56.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.168+0000 7fb2b9e00640 1 -- 172.21.15.47:0/3720870309 learned_addr learned my addr 172.21.15.47:0/3720870309 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.169+0000 7fb2b9e00640 1 -- 172.21.15.47:0/3720870309 >> v2:172.21.15.113:3300/0 conn(0x7fb2bc0c1a00 msgr2=0x7fb2bc0c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.169+0000 7fb2b9e00640 1 --2- 172.21.15.47:0/3720870309 >> v2:172.21.15.113:3300/0 conn(0x7fb2bc0c1a00 0x7fb2bc0c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.169+0000 7fb2b9e00640 1 -- 172.21.15.47:0/3720870309 >> v1:172.21.15.113:6789/0 conn(0x7fb2bc072f60 legacy=0x7fb2bc0733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.169+0000 7fb2b9e00640 1 -- 172.21.15.47:0/3720870309 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc15b2d0 con 0x7fb2bc151780 2024-04-24T22:42:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.170+0000 7fb2b9e00640 1 --2- 172.21.15.47:0/3720870309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc151780 0x7fb2bc0c8590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0705a0 tx=0x7fb2a802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19df61b3c991b639 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.173+0000 7fb2b8a00640 1 -- 172.21.15.47:0/3720870309 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8037070 con 0x7fb2bc151780 2024-04-24T22:42:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.173+0000 7fb2b8a00640 1 -- 172.21.15.47:0/3720870309 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a8032440 con 0x7fb2bc151780 2024-04-24T22:42:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.173+0000 7fb2b8a00640 1 -- 172.21.15.47:0/3720870309 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8032810 con 0x7fb2bc151780 2024-04-24T22:42:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.174+0000 7fb2c0a00640 1 -- 172.21.15.47:0/3720870309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc151780 msgr2=0x7fb2bc0c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.174+0000 7fb2c0a00640 1 --2- 172.21.15.47:0/3720870309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc151780 0x7fb2bc0c8590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0705a0 tx=0x7fb2a802a2c0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.176+0000 7fb2c0a00640 1 -- 172.21.15.47:0/3720870309 shutdown_connections 2024-04-24T22:42:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.176+0000 7fb2c0a00640 1 --2- 172.21.15.47:0/3720870309 >> v2:172.21.15.113:3300/0 conn(0x7fb2bc0c1a00 0x7fb2bc0c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.176+0000 7fb2c0a00640 1 --2- 172.21.15.47:0/3720870309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc151780 0x7fb2bc0c8590 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.176+0000 7fb2c0a00640 1 -- 172.21.15.47:0/3720870309 >> 172.21.15.47:0/3720870309 conn(0x7fb2bc06e580 msgr2=0x7fb2bc06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.176+0000 7fb2c0a00640 1 -- 172.21.15.47:0/3720870309 shutdown_connections 2024-04-24T22:42:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.176+0000 7fb2c0a00640 1 -- 172.21.15.47:0/3720870309 wait complete. 2024-04-24T22:42:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.179+0000 7fb2c0a00640 1 Processor -- start 2024-04-24T22:42:56.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.179+0000 7fb2c0a00640 1 -- start start 2024-04-24T22:42:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.180+0000 7fb2c0a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2bc072f60 0x7fb2bc158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.180+0000 7fb2c0a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc0c1a00 0x7fb2bc1eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.180+0000 7fb2c0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2bc151780 0x7fb2bc144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.180+0000 7fb2c0a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc157870 con 0x7fb2bc0c1a00 2024-04-24T22:42:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.180+0000 7fb2c0a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1576f0 con 0x7fb2bc151780 2024-04-24T22:42:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.180+0000 7fb2c0a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc1579f0 con 0x7fb2bc072f60 2024-04-24T22:42:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.181+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2bc072f60 0x7fb2bc158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.181+0000 7fb2b9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2bc072f60 0x7fb2bc158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:50710/0 (socket says 172.21.15.47:50710) 2024-04-24T22:42:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.181+0000 7fb2b9e00640 1 -- 172.21.15.47:0/738300605 learned_addr learned my addr 172.21.15.47:0/738300605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.181+0000 7fb2b9400640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc0c1a00 0x7fb2bc1eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.181+0000 7fb2ba800640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2bc151780 0x7fb2bc144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.181+0000 7fb2b9e00640 1 -- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2bc072f60 msgr2=0x7fb2bc158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-24T22:42:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.181+0000 7fb2b9e00640 1 -- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2bc072f60 msgr2=0x7fb2bc158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:42:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.181+0000 7fb2b9e00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2bc072f60 0x7fb2bc158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:42:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.181+0000 7fb2b9e00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2bc072f60 0x7fb2bc158460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:42:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.182+0000 7fb2ba800640 1 -- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2bc072f60 msgr2=0x7fb2bc158460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:42:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.182+0000 7fb2ba800640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2bc072f60 0x7fb2bc158460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.182+0000 7fb2ba800640 1 -- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc0c1a00 msgr2=0x7fb2bc1eccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.182+0000 7fb2ba800640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc0c1a00 0x7fb2bc1eccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.182+0000 7fb2ba800640 1 -- 172.21.15.47:0/738300605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc1ed210 con 0x7fb2bc151780 2024-04-24T22:42:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.182+0000 7fb2ba800640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2bc151780 0x7fb2bc144d60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400c780 tx=0x7fb2b400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.183+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4016020 con 0x7fb2bc151780 2024-04-24T22:42:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.183+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b400aa50 con 0x7fb2bc151780 2024-04-24T22:42:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.183+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400ae40 con 0x7fb2bc151780 2024-04-24T22:42:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.183+0000 7fb2c0a00640 1 -- 172.21.15.47:0/738300605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc13f600 con 0x7fb2bc151780 2024-04-24T22:42:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.184+0000 7fb2c0a00640 1 -- 172.21.15.47:0/738300605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc13fbc0 con 0x7fb2bc151780 2024-04-24T22:42:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.187+0000 7fb2c0a00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ef830 0x7fb2bc159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.187+0000 7fb2c0a00640 1 -- 172.21.15.47:0/738300605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc158a00 con 0x7fb2bc1ef830 2024-04-24T22:42:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.188+0000 7fb2b9e00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ef830 0x7fb2bc159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.188+0000 7fb2b9e00640 1 -- 172.21.15.47:0/738300605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2bc073b90 con 0x7fb2bc1ef830 2024-04-24T22:42:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.189+0000 7fb2b9e00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ef830 0x7fb2bc159de0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2a802a300 tx=0x7fb2a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.189+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8041070 con 0x7fb2bc1ef830 2024-04-24T22:42:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.190+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb2a803c660 con 0x7fb2bc1ef830 2024-04-24T22:42:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.190+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ef830 msgr2=0x7fb2bc159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.190+0000 7fb2b2000640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ef830 0x7fb2bc159de0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2a802a300 tx=0x7fb2a80046c0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.190+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ef830 msgr2=0x7fb2bc159de0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.190+0000 7fb2b2000640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ef830 0x7fb2bc159de0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2a802a300 tx=0x7fb2a80046c0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.190+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2b4019020 con 0x7fb2bc151780 2024-04-24T22:42:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.191+0000 7fb2b2000640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb29c06bb00 0x7fb29c06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.191+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb2b408ac50 con 0x7fb2bc151780 2024-04-24T22:42:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.192+0000 7fb2b9e00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb29c06bb00 0x7fb29c06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.196+0000 7fb2b9e00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb29c06bb00 0x7fb29c06dfc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80090b0 tx=0x7fb2a8048040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.207+0000 7fb2c0a00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ec2c0 0x7fb2bc15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.207+0000 7fb2c0a00640 1 -- 172.21.15.47:0/738300605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc0b1ce0 con 0x7fb2bc1ec2c0 2024-04-24T22:42:56.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.208+0000 7fb2b9400640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ec2c0 0x7fb2bc15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.209+0000 7fb2b9400640 1 -- 172.21.15.47:0/738300605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb2bc0c8320 con 0x7fb2bc1ec2c0 2024-04-24T22:42:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.210+0000 7fb2b9400640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ec2c0 0x7fb2bc15a320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0c1de0 tx=0x7fb2ac013580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.210+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac01d030 con 0x7fb2bc1ec2c0 2024-04-24T22:42:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.210+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5040 (secure 0 0 0) 0x7fb2ac013c80 con 0x7fb2bc1ec2c0 2024-04-24T22:42:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.210+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ec2c0 msgr2=0x7fb2bc15a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.210+0000 7fb2b2000640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ec2c0 0x7fb2bc15a320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0c1de0 tx=0x7fb2ac013580 comp rx=0 tx=0).stop 2024-04-24T22:42:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.210+0000 7fb2b2000640 1 -- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ec2c0 msgr2=0x7fb2bc15a320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:56.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.210+0000 7fb2b2000640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2bc1ec2c0 0x7fb2bc15a320 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0c1de0 tx=0x7fb2ac013580 comp rx=0 tx=0).stop 2024-04-24T22:42:56.212 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 30, 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 116247, 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.213 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:56.214 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:56.215 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:42:56.216 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:56.217 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:56.218 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:56.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:42:56.220 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:56.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:42:56.222 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.223 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:42:56.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:42:56.225 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:42:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.211+0000 7fb2c0a00640 1 -- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb29c06bb00 msgr2=0x7fb29c06dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.211+0000 7fb2c0a00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb29c06bb00 0x7fb29c06dfc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80090b0 tx=0x7fb2a8048040 comp rx=0 tx=0).stop 2024-04-24T22:42:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.211+0000 7fb2ba800640 1 -- 172.21.15.47:0/738300605 reap_dead start 2024-04-24T22:42:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.211+0000 7fb2c0a00640 1 -- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2bc151780 msgr2=0x7fb2bc144d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.211+0000 7fb2c0a00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2bc151780 0x7fb2bc144d60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400c780 tx=0x7fb2b400cc40 comp rx=0 tx=0).stop 2024-04-24T22:42:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.211+0000 7fb2c0a00640 1 -- 172.21.15.47:0/738300605 shutdown_connections 2024-04-24T22:42:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.212+0000 7fb2c0a00640 1 --2- 172.21.15.47:0/738300605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2bc151780 0x7fb2bc144d60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.212+0000 7fb2c0a00640 1 -- 172.21.15.47:0/738300605 >> 172.21.15.47:0/738300605 conn(0x7fb2bc06e580 msgr2=0x7fb2bc0c0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.212+0000 7fb2c0a00640 1 -- 172.21.15.47:0/738300605 shutdown_connections 2024-04-24T22:42:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.212+0000 7fb2c0a00640 1 -- 172.21.15.47:0/738300605 wait complete. 2024-04-24T22:42:56.236 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:42:56.236 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-24T22:42:56.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:42:56.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.445+0000 7f40d0400640 1 Processor -- start 2024-04-24T22:42:56.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.445+0000 7f40d0400640 1 -- start start 2024-04-24T22:42:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.446+0000 7f40d0400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40c815a680 0x7f40c80c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.446+0000 7f40d0400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40c80c8ad0 0x7f40c80c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.446+0000 7f40d0400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40c815ab10 con 0x7f40c8154960 2024-04-24T22:42:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.446+0000 7f40d0400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f40c815adf0 con 0x7f40c80c8ad0 2024-04-24T22:42:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.446+0000 7f40d0400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f40c815b0d0 con 0x7f40c815a680 2024-04-24T22:42:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.446+0000 7f40cd000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40c815a680 0x7f40c80c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.447+0000 7f40cda00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f40c8154960 0x7f40c815e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:33218/0 (socket says 172.21.15.47:33218) 2024-04-24T22:42:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.447+0000 7f40cda00640 1 -- 172.21.15.47:0/405359165 learned_addr learned my addr 172.21.15.47:0/405359165 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:42:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.447+0000 7f40ce400640 1 --2- 172.21.15.47:0/405359165 >> v2:172.21.15.47:3300/0 conn(0x7f40c80c8ad0 0x7f40c80c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.447+0000 7f40cd000640 1 -- 172.21.15.47:0/405359165 >> v2:172.21.15.47:3300/0 conn(0x7f40c80c8ad0 msgr2=0x7f40c80c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.447+0000 7f40cd000640 1 --2- 172.21.15.47:0/405359165 >> v2:172.21.15.47:3300/0 conn(0x7f40c80c8ad0 0x7f40c80c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.447+0000 7f40cd000640 1 -- 172.21.15.47:0/405359165 >> v1:172.21.15.113:6789/0 conn(0x7f40c8154960 legacy=0x7f40c815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.447+0000 7f40cd000640 1 -- 172.21.15.47:0/405359165 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c815b3b0 con 0x7f40c815a680 2024-04-24T22:42:56.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.447+0000 7f40ce400640 1 --2- 172.21.15.47:0/405359165 >> v2:172.21.15.47:3300/0 conn(0x7f40c80c8ad0 0x7f40c80c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.448+0000 7f40cd000640 1 --2- 172.21.15.47:0/405359165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 0x7f40c80c8590 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f40c80c2ed0 tx=0x7f40b402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9463cc1f764f835c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.448+0000 7f40c7e00640 1 -- 172.21.15.47:0/405359165 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4037070 con 0x7f40c815a680 2024-04-24T22:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.448+0000 7f40c7e00640 1 -- 172.21.15.47:0/405359165 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b4032440 con 0x7f40c815a680 2024-04-24T22:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.448+0000 7f40c7e00640 1 -- 172.21.15.47:0/405359165 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4032810 con 0x7f40c815a680 2024-04-24T22:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.449+0000 7f40d0400640 1 -- 172.21.15.47:0/405359165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 msgr2=0x7f40c80c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.449+0000 7f40d0400640 1 --2- 172.21.15.47:0/405359165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 0x7f40c80c8590 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f40c80c2ed0 tx=0x7f40b402a1a0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.449+0000 7f40d0400640 1 -- 172.21.15.47:0/405359165 shutdown_connections 2024-04-24T22:42:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.449+0000 7f40d0400640 1 --2- 172.21.15.47:0/405359165 >> v2:172.21.15.47:3300/0 conn(0x7f40c80c8ad0 0x7f40c80c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.449+0000 7f40d0400640 1 --2- 172.21.15.47:0/405359165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 0x7f40c80c8590 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.449+0000 7f40d0400640 1 -- 172.21.15.47:0/405359165 >> 172.21.15.47:0/405359165 conn(0x7f40c806e580 msgr2=0x7f40c806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.449+0000 7f40d0400640 1 -- 172.21.15.47:0/405359165 shutdown_connections 2024-04-24T22:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.449+0000 7f40d0400640 1 -- 172.21.15.47:0/405359165 wait complete. 2024-04-24T22:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.451+0000 7f40d0400640 1 Processor -- start 2024-04-24T22:42:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.451+0000 7f40d0400640 1 -- start start 2024-04-24T22:42:56.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.451+0000 7f40d0400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c80c8ad0 0x7f40c814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40d0400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c8154960 0x7f40c815ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40d0400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c814d440 0x7f40c815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40d0400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f40c815ea00 con 0x7f40c8154960 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40d0400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8154d40 con 0x7f40c80c8ad0 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40d0400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f40c815eb80 con 0x7f40c814d440 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40cda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c80c8ad0 0x7f40c814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40cda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c80c8ad0 0x7f40c814cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:58930/0 (socket says 172.21.15.47:58930) 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40cda00640 1 -- 172.21.15.47:0/1972423820 learned_addr learned my addr 172.21.15.47:0/1972423820 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40ce400640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c814d440 0x7f40c815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40cd000640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c8154960 0x7f40c815ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40cda00640 1 -- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c814d440 msgr2=0x7f40c815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40cda00640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c814d440 0x7f40c815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40cda00640 1 -- 172.21.15.47:0/1972423820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c8154960 msgr2=0x7f40c815ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40cda00640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c8154960 0x7f40c815ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.452+0000 7f40cda00640 1 -- 172.21.15.47:0/1972423820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c81f0050 con 0x7f40c80c8ad0 2024-04-24T22:42:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.453+0000 7f40cda00640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c80c8ad0 0x7f40c814cf00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00cbd0 tx=0x7f40bc00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.453+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc017070 con 0x7f40c80c8ad0 2024-04-24T22:42:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.453+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40bc00ac70 con 0x7f40c80c8ad0 2024-04-24T22:42:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.453+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc012600 con 0x7f40c80c8ad0 2024-04-24T22:42:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.453+0000 7f40d0400640 1 -- 172.21.15.47:0/1972423820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c81f37d0 con 0x7f40c80c8ad0 2024-04-24T22:42:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.453+0000 7f40d0400640 1 -- 172.21.15.47:0/1972423820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c81f3d90 con 0x7f40c80c8ad0 2024-04-24T22:42:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.454+0000 7f40d0400640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 0x7f40c81f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.454+0000 7f40d0400640 1 -- 172.21.15.47:0/1972423820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8156e40 con 0x7f40c815a680 2024-04-24T22:42:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.454+0000 7f40cd000640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 0x7f40c81f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.455+0000 7f40cd000640 1 -- 172.21.15.47:0/1972423820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40c8073b90 con 0x7f40c815a680 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.455+0000 7f40cd000640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 0x7f40c81f3fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f40c806f540 tx=0x7f40b40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.455+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40bc01a070 con 0x7f40c80c8ad0 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.456+0000 7f40c5600640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f40a806be30 0x7f40a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.456+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f40bc08ac50 con 0x7f40c80c8ad0 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.456+0000 7f40ce400640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f40a806be30 0x7f40a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.456+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4043070 con 0x7f40c815a680 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.456+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f40b403c650 con 0x7f40c815a680 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.456+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 msgr2=0x7f40c81f3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.456+0000 7f40c5600640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 0x7f40c81f3fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f40c806f540 tx=0x7f40b40046c0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.456+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 msgr2=0x7f40c81f3fd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.456+0000 7f40c5600640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c815a680 0x7f40c81f3fd0 secure :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f40c806f540 tx=0x7f40b40046c0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.457+0000 7f40ce400640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f40a806be30 0x7f40a806e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009b60 tx=0x7f40c0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.463+0000 7f40d0400640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c8157bf0 0x7f40c81f0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.463+0000 7f40d0400640 1 -- 172.21.15.47:0/1972423820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f40c80c76d0 con 0x7f40c8157bf0 2024-04-24T22:42:56.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.463+0000 7f40cd000640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c8157bf0 0x7f40c81f0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.464+0000 7f40cd000640 1 -- 172.21.15.47:0/1972423820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f40c8151eb0 con 0x7f40c8157bf0 2024-04-24T22:42:56.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.464+0000 7f40cd000640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c8157bf0 0x7f40c81f0550 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f40b402a1e0 tx=0x7f40b4008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.464+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4037040 con 0x7f40c8157bf0 2024-04-24T22:42:56.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.464+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5065 (secure 0 0 0) 0x7f40b403c800 con 0x7f40c8157bf0 2024-04-24T22:42:56.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.464+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c8157bf0 msgr2=0x7f40c81f0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.464+0000 7f40c5600640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c8157bf0 0x7f40c81f0550 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f40b402a1e0 tx=0x7f40b4008030 comp rx=0 tx=0).stop 2024-04-24T22:42:56.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.464+0000 7f40c5600640 1 -- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c8157bf0 msgr2=0x7f40c81f0550 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:56.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.464+0000 7f40c5600640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c8157bf0 0x7f40c81f0550 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f40b402a1e0 tx=0x7f40b4008030 comp rx=0 tx=0).stop 2024-04-24T22:42:56.467 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:42:56.467 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:56.467 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:56.467 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:42:56.467 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 30, 2024-04-24T22:42:56.467 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 218032, 2024-04-24T22:42:56.467 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:56.468 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:56.469 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:42:56.470 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:56.471 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:42:56.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:56.473 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:56.474 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.475 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:56.476 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:56.477 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.478 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.479 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-24T22:42:56.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.481 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.481 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:42:56.481 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:42:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.465+0000 7f40d0400640 1 -- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f40a806be30 msgr2=0x7f40a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.465+0000 7f40d0400640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f40a806be30 0x7f40a806e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009b60 tx=0x7f40c0009190 comp rx=0 tx=0).stop 2024-04-24T22:42:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.465+0000 7f40d0400640 1 -- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c80c8ad0 msgr2=0x7f40c814cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.465+0000 7f40d0400640 1 --2- 172.21.15.47:0/1972423820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c80c8ad0 0x7f40c814cf00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00cbd0 tx=0x7f40bc00a060 comp rx=0 tx=0).stop 2024-04-24T22:42:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.465+0000 7f40ce400640 1 -- 172.21.15.47:0/1972423820 reap_dead start 2024-04-24T22:42:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.465+0000 7f40ce400640 1 -- 172.21.15.47:0/1972423820 reap_dead start 2024-04-24T22:42:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.465+0000 7f40d0400640 1 -- 172.21.15.47:0/1972423820 shutdown_connections 2024-04-24T22:42:56.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.465+0000 7f40d0400640 1 -- 172.21.15.47:0/1972423820 >> 172.21.15.47:0/1972423820 conn(0x7f40c806e580 msgr2=0x7f40c8061c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.465+0000 7f40d0400640 1 -- 172.21.15.47:0/1972423820 shutdown_connections 2024-04-24T22:42:56.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.465+0000 7f40d0400640 1 -- 172.21.15.47:0/1972423820 wait complete. 2024-04-24T22:42:56.483 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:42:56.484 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-24T22:42:56.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-24T22:42:56.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.752+0000 7fa4f0600640 1 Processor -- start 2024-04-24T22:42:56.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.752+0000 7fa4f0600640 1 -- start start 2024-04-24T22:42:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.755+0000 7fa4f0600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4e8158fc0 0x7fa4e80c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.755+0000 7fa4f0600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4e81621d0 con 0x7fa4e8158be0 2024-04-24T22:42:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.755+0000 7fa4f0600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4e81624b0 con 0x7fa4e80c1280 2024-04-24T22:42:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.755+0000 7fa4f0600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e8162790 con 0x7fa4e8158fc0 2024-04-24T22:42:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.755+0000 7fa4ed200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4e8158fc0 0x7fa4e80c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.755+0000 7fa4ed200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4e8158fc0 0x7fa4e80c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:58968/0 (socket says 172.21.15.47:58968) 2024-04-24T22:42:56.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.755+0000 7fa4ed200640 1 -- 172.21.15.47:0/1957858786 learned_addr learned my addr 172.21.15.47:0/1957858786 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:56.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.756+0000 7fa4ed200640 1 -- 172.21.15.47:0/1957858786 >> v1:172.21.15.113:6789/0 conn(0x7fa4e80c1280 legacy=0x7fa4e815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:42:56.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.757+0000 7fa4e7e00640 1 -- 172.21.15.47:0/1957858786 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790625584 0 0) 0x7fa4e81621d0 con 0x7fa4e8158be0 2024-04-24T22:42:56.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.757+0000 7fa4ed200640 1 -- 172.21.15.47:0/1957858786 >> v1:172.21.15.47:6789/0 conn(0x7fa4e8158be0 legacy=0x7fa4e80c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.757+0000 7fa4ed200640 1 -- 172.21.15.47:0/1957858786 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e8162a70 con 0x7fa4e8158fc0 2024-04-24T22:42:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.758+0000 7fa4ed200640 1 --2- 172.21.15.47:0/1957858786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e8158fc0 0x7fa4e80c8590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009870 tx=0x7fa4d402a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90b6ea8c6ee81219 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.759+0000 7fa4e7e00640 1 -- 172.21.15.47:0/1957858786 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4037070 con 0x7fa4e8158fc0 2024-04-24T22:42:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.759+0000 7fa4e7e00640 1 -- 172.21.15.47:0/1957858786 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d4032440 con 0x7fa4e8158fc0 2024-04-24T22:42:56.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.759+0000 7fa4e7e00640 1 -- 172.21.15.47:0/1957858786 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4032810 con 0x7fa4e8158fc0 2024-04-24T22:42:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.760+0000 7fa4f0600640 1 -- 172.21.15.47:0/1957858786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e8158fc0 msgr2=0x7fa4e80c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.760+0000 7fa4f0600640 1 --2- 172.21.15.47:0/1957858786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e8158fc0 0x7fa4e80c8590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009870 tx=0x7fa4d402a2a0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.760+0000 7fa4f0600640 1 -- 172.21.15.47:0/1957858786 shutdown_connections 2024-04-24T22:42:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.760+0000 7fa4f0600640 1 --2- 172.21.15.47:0/1957858786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e8158fc0 0x7fa4e80c8590 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.760+0000 7fa4f0600640 1 -- 172.21.15.47:0/1957858786 >> 172.21.15.47:0/1957858786 conn(0x7fa4e806e580 msgr2=0x7fa4e806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:56.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.760+0000 7fa4f0600640 1 -- 172.21.15.47:0/1957858786 shutdown_connections 2024-04-24T22:42:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.761+0000 7fa4f0600640 1 -- 172.21.15.47:0/1957858786 wait complete. 2024-04-24T22:42:56.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.763+0000 7fa4f0600640 1 Processor -- start 2024-04-24T22:42:56.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.763+0000 7fa4f0600640 1 -- start start 2024-04-24T22:42:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.765+0000 7fa4f0600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e80c1280 0x7fa4e81fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.765+0000 7fa4f0600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8158be0 0x7fa4e8155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.765+0000 7fa4f0600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e8158fc0 0x7fa4e8071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.765+0000 7fa4f0600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e815f460 con 0x7fa4e8158fc0 2024-04-24T22:42:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.765+0000 7fa4f0600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e815f2e0 con 0x7fa4e80c1280 2024-04-24T22:42:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.765+0000 7fa4f0600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e81567d0 con 0x7fa4e8158be0 2024-04-24T22:42:56.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.765+0000 7fa4edc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e80c1280 0x7fa4e81fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.765+0000 7fa4ee600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e8158fc0 0x7fa4e8071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.766+0000 7fa4ee600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e8158fc0 0x7fa4e8071f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40828/0 (socket says 172.21.15.47:40828) 2024-04-24T22:42:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.766+0000 7fa4ee600640 1 -- 172.21.15.47:0/1999291716 learned_addr learned my addr 172.21.15.47:0/1999291716 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.766+0000 7fa4ed200640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8158be0 0x7fa4e8155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.766+0000 7fa4ee600640 1 -- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8158be0 msgr2=0x7fa4e8155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.766+0000 7fa4ee600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8158be0 0x7fa4e8155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.766+0000 7fa4ee600640 1 -- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e80c1280 msgr2=0x7fa4e81fce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.766+0000 7fa4ee600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e80c1280 0x7fa4e81fce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.766+0000 7fa4ee600640 1 -- 172.21.15.47:0/1999291716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e8156ab0 con 0x7fa4e8158fc0 2024-04-24T22:42:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.767+0000 7fa4ee600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e8158fc0 0x7fa4e8071f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000b410 tx=0x7fa4e000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.767+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0015030 con 0x7fa4e8158fc0 2024-04-24T22:42:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.767+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e0004020 con 0x7fa4e8158fc0 2024-04-24T22:42:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.767+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0004410 con 0x7fa4e8158fc0 2024-04-24T22:42:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.767+0000 7fa4f0600640 1 -- 172.21.15.47:0/1999291716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e8200580 con 0x7fa4e8158fc0 2024-04-24T22:42:56.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.767+0000 7fa4f0600640 1 -- 172.21.15.47:0/1999291716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e8200b40 con 0x7fa4e8158fc0 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.770+0000 7fa4f0600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8201060 0x7fa4e81602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.770+0000 7fa4f0600640 1 -- 172.21.15.47:0/1999291716 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e815f5e0 con 0x7fa4e8201060 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.771+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4e0018070 con 0x7fa4e8158fc0 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.772+0000 7fa4e5600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4c806bf00 0x7fa4c806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.773+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa4e008a880 con 0x7fa4e8158fc0 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.773+0000 7fa4ed200640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4c806bf00 0x7fa4c806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.774+0000 7fa4edc00640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8201060 0x7fa4e81602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.774+0000 7fa4ed200640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4c806bf00 0x7fa4c806e3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4034000 tx=0x7fa4d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.774+0000 7fa4edc00640 1 -- 172.21.15.47:0/1999291716 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4e82003a0 con 0x7fa4e8201060 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.775+0000 7fa4edc00640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8201060 0x7fa4e81602d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc002230 tx=0x7fa4dc00fd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.775+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc003b80 con 0x7fa4e8201060 2024-04-24T22:42:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.775+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa4dc018070 con 0x7fa4e8201060 2024-04-24T22:42:56.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.775+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8201060 msgr2=0x7fa4e81602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.775+0000 7fa4e5600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8201060 0x7fa4e81602d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc002230 tx=0x7fa4dc00fd40 comp rx=0 tx=0).stop 2024-04-24T22:42:56.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.775+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8201060 msgr2=0x7fa4e81602d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:56.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.775+0000 7fa4e5600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e8201060 0x7fa4e81602d0 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc002230 tx=0x7fa4dc00fd40 comp rx=0 tx=0).stop 2024-04-24T22:42:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.786+0000 7fa4f0600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e81593d0 0x7fa4e8160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.786+0000 7fa4f0600640 1 -- 172.21.15.47:0/1999291716 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e8073b90 con 0x7fa4e81593d0 2024-04-24T22:42:56.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.787+0000 7fa4edc00640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e81593d0 0x7fa4e8160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:56.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.788+0000 7fa4edc00640 1 -- 172.21.15.47:0/1999291716 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa4e80c7100 con 0x7fa4e81593d0 2024-04-24T22:42:56.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.791+0000 7fa4edc00640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e81593d0 0x7fa4e8160810 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc002230 tx=0x7fa4dc001400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:56.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.791+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00d9b0 con 0x7fa4e81593d0 2024-04-24T22:42:56.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.792+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4554 (secure 0 0 0) 0x7fa4dc00d9b0 con 0x7fa4e81593d0 2024-04-24T22:42:56.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.792+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e81593d0 msgr2=0x7fa4e8160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.792+0000 7fa4e5600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e81593d0 0x7fa4e8160810 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc002230 tx=0x7fa4dc001400 comp rx=0 tx=0).stop 2024-04-24T22:42:56.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.792+0000 7fa4e5600640 1 -- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e81593d0 msgr2=0x7fa4e8160810 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:42:56.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.792+0000 7fa4e5600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e81593d0 0x7fa4e8160810 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc002230 tx=0x7fa4dc001400 comp rx=0 tx=0).stop 2024-04-24T22:42:56.794 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:42:56.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:42:56.794 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:42:56.794 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:42:56.794 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 30, 2024-04-24T22:42:56.794 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 10432, 2024-04-24T22:42:56.794 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:42:56.794 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:42:56.794 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:56.795 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:56.796 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:42:56.797 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:42:56.798 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:42:56.800 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:42:56.802 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:42:56.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:42:56.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:42:56.805 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:42:56.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.793+0000 7fa4f0600640 1 -- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4c806bf00 msgr2=0x7fa4c806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.793+0000 7fa4f0600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4c806bf00 0x7fa4c806e3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4034000 tx=0x7fa4d402e000 comp rx=0 tx=0).stop 2024-04-24T22:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.793+0000 7fa4f0600640 1 -- 172.21.15.47:0/1999291716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e8158fc0 msgr2=0x7fa4e8071f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.793+0000 7fa4f0600640 1 --2- 172.21.15.47:0/1999291716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e8158fc0 0x7fa4e8071f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000b410 tx=0x7fa4e000b8d0 comp rx=0 tx=0).stop 2024-04-24T22:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.793+0000 7fa4ee600640 1 -- 172.21.15.47:0/1999291716 reap_dead start 2024-04-24T22:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.793+0000 7fa4ee600640 1 -- 172.21.15.47:0/1999291716 reap_dead start 2024-04-24T22:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.794+0000 7fa4f0600640 1 -- 172.21.15.47:0/1999291716 shutdown_connections 2024-04-24T22:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.794+0000 7fa4f0600640 1 -- 172.21.15.47:0/1999291716 >> 172.21.15.47:0/1999291716 conn(0x7fa4e806e580 msgr2=0x7fa4e80c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.794+0000 7fa4f0600640 1 -- 172.21.15.47:0/1999291716 shutdown_connections 2024-04-24T22:42:56.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:56.794+0000 7fa4f0600640 1 -- 172.21.15.47:0/1999291716 wait complete. 2024-04-24T22:42:56.812 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:42:56.812 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-24T22:42:56.812 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-24T22:42:57.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.093+0000 7f2c05800640 1 Processor -- start 2024-04-24T22:42:57.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.094+0000 7f2c05800640 1 -- start start 2024-04-24T22:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.097+0000 7f2c05800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c00150c10 0x7f2c000c8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.097+0000 7f2c05800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000c7df0 0x7f2c000cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.097+0000 7f2c05800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c000cc6e0 0x7f2c000720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.097+0000 7f2c05800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2c00156d00 con 0x7f2c000cc6e0 2024-04-24T22:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.097+0000 7f2c05800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2c00156fe0 con 0x7f2c00150c10 2024-04-24T22:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.097+0000 7f2c05800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2c001572c0 con 0x7f2c000c7df0 2024-04-24T22:42:57.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.097+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000c7df0 0x7f2c000cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:57.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.097+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000c7df0 0x7f2c000cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:50768/0 (socket says 172.21.15.47:50768) 2024-04-24T22:42:57.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.097+0000 7f2bfe000640 1 -- 172.21.15.47:0/2975230926 learned_addr learned my addr 172.21.15.47:0/2975230926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:57.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.098+0000 7f2bfe000640 1 -- 172.21.15.47:0/2975230926 >> v2:172.21.15.113:3300/0 conn(0x7f2c00150c10 msgr2=0x7f2c000c8360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:57.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.098+0000 7f2bfe000640 1 --2- 172.21.15.47:0/2975230926 >> v2:172.21.15.113:3300/0 conn(0x7f2c00150c10 0x7f2c000c8360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.098+0000 7f2bfe000640 1 -- 172.21.15.47:0/2975230926 >> v2:172.21.15.47:3300/0 conn(0x7f2c000cc6e0 msgr2=0x7f2c000720c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:42:57.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.098+0000 7f2bfe000640 1 --2- 172.21.15.47:0/2975230926 >> v2:172.21.15.47:3300/0 conn(0x7f2c000cc6e0 0x7f2c000720c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.098+0000 7f2bfe000640 1 -- 172.21.15.47:0/2975230926 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c001575a0 con 0x7f2c000c7df0 2024-04-24T22:42:57.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.103+0000 7f2bfe000640 1 --2- 172.21.15.47:0/2975230926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000c7df0 0x7f2c000cc1a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2c000ccf40 tx=0x7f2bf002a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43238eb31a4e2435 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:57.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.104+0000 7f2bfd600640 1 -- 172.21.15.47:0/2975230926 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0037070 con 0x7f2c000c7df0 2024-04-24T22:42:57.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.104+0000 7f2bfd600640 1 -- 172.21.15.47:0/2975230926 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf0032440 con 0x7f2c000c7df0 2024-04-24T22:42:57.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.104+0000 7f2bfd600640 1 -- 172.21.15.47:0/2975230926 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0032810 con 0x7f2c000c7df0 2024-04-24T22:42:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.105+0000 7f2c05800640 1 -- 172.21.15.47:0/2975230926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000c7df0 msgr2=0x7f2c000cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.105+0000 7f2c05800640 1 --2- 172.21.15.47:0/2975230926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000c7df0 0x7f2c000cc1a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2c000ccf40 tx=0x7f2bf002a2a0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.106+0000 7f2c05800640 1 -- 172.21.15.47:0/2975230926 shutdown_connections 2024-04-24T22:42:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.106+0000 7f2c05800640 1 --2- 172.21.15.47:0/2975230926 >> v2:172.21.15.47:3300/0 conn(0x7f2c000cc6e0 0x7f2c000720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.106+0000 7f2c05800640 1 --2- 172.21.15.47:0/2975230926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000c7df0 0x7f2c000cc1a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.106+0000 7f2c05800640 1 --2- 172.21.15.47:0/2975230926 >> v2:172.21.15.113:3300/0 conn(0x7f2c00150c10 0x7f2c000c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.106+0000 7f2c05800640 1 -- 172.21.15.47:0/2975230926 >> 172.21.15.47:0/2975230926 conn(0x7f2c0006b160 msgr2=0x7f2c0006b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.106+0000 7f2c05800640 1 -- 172.21.15.47:0/2975230926 shutdown_connections 2024-04-24T22:42:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.107+0000 7f2c05800640 1 -- 172.21.15.47:0/2975230926 wait complete. 2024-04-24T22:42:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.110+0000 7f2c05800640 1 Processor -- start 2024-04-24T22:42:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.110+0000 7f2c05800640 1 -- start start 2024-04-24T22:42:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.112+0000 7f2c05800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c000c7df0 0x7f2c001f0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.113+0000 7f2c05800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000cc6e0 0x7f2c0014dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.113+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c000c7df0 0x7f2c001f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.113+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c000c7df0 0x7f2c001f0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40842/0 (socket says 172.21.15.47:40842) 2024-04-24T22:42:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.113+0000 7f2bfea00640 1 -- 172.21.15.47:0/1908691712 learned_addr learned my addr 172.21.15.47:0/1908691712 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:42:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.114+0000 7f2c05800640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c00150c10 0x7f2c0014bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.114+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2c001569d0 con 0x7f2c000c7df0 2024-04-24T22:42:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.114+0000 7f2bfe000640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000cc6e0 0x7f2c0014dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.114+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00071ba0 con 0x7f2c00150c10 2024-04-24T22:42:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.114+0000 7f2bff400640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c00150c10 0x7f2c0014bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.115+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2c000c88a0 con 0x7f2c000cc6e0 2024-04-24T22:42:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.116+0000 7f2bff400640 1 -- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000cc6e0 msgr2=0x7f2c0014dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.116+0000 7f2bff400640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000cc6e0 0x7f2c0014dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.116+0000 7f2bff400640 1 -- 172.21.15.47:0/1908691712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c000c7df0 msgr2=0x7f2c001f0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.116+0000 7f2bff400640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c000c7df0 0x7f2c001f0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.116+0000 7f2bff400640 1 -- 172.21.15.47:0/1908691712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c001f0eb0 con 0x7f2c00150c10 2024-04-24T22:42:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.116+0000 7f2bfea00640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c000c7df0 0x7f2c001f0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:42:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.117+0000 7f2bff400640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c00150c10 0x7f2c0014bf20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800be30 tx=0x7f2bf800bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.118+0000 7f2bf6a00640 1 -- 172.21.15.47:0/1908691712 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf801b070 con 0x7f2c00150c10 2024-04-24T22:42:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.118+0000 7f2bf6a00640 1 -- 172.21.15.47:0/1908691712 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf8016440 con 0x7f2c00150c10 2024-04-24T22:42:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.118+0000 7f2bf6a00640 1 -- 172.21.15.47:0/1908691712 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8016870 con 0x7f2c00150c10 2024-04-24T22:42:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.121+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c00154de0 con 0x7f2c00150c10 2024-04-24T22:42:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.121+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c000cc070 con 0x7f2c00150c10 2024-04-24T22:42:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.123+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c00073b90 con 0x7f2c00150c10 2024-04-24T22:42:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.128+0000 7f2bf6a00640 1 -- 172.21.15.47:0/1908691712 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2bf8003a30 con 0x7f2c00150c10 2024-04-24T22:42:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.130+0000 7f2bf6a00640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2be006bf30 0x7f2be006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:42:57.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.130+0000 7f2bf6a00640 1 -- 172.21.15.47:0/1908691712 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f2bf808d150 con 0x7f2c00150c10 2024-04-24T22:42:57.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.131+0000 7f2bfea00640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2be006bf30 0x7f2be006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:42:57.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.131+0000 7f2bfea00640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2be006bf30 0x7f2be006e3f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009dc0 tx=0x7f2bec0090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:42:57.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.132+0000 7f2bf6a00640 1 -- 172.21.15.47:0/1908691712 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2bf805b410 con 0x7f2c00150c10 2024-04-24T22:42:57.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.563+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f2c000cc4c0 con 0x7f2c00150c10 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.568+0000 7f2bf6a00640 1 -- 172.21.15.47:0/1908691712 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f2bf805f710 con 0x7f2c00150c10 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.572+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2be006bf30 msgr2=0x7f2be006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.572+0000 7f2c05800640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2be006bf30 0x7f2be006e3f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009dc0 tx=0x7f2bec0090e0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.572+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c00150c10 msgr2=0x7f2c0014bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.572+0000 7f2c05800640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c00150c10 0x7f2c0014bf20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800be30 tx=0x7f2bf800bf20 comp rx=0 tx=0).stop 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.573+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 shutdown_connections 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.573+0000 7f2c05800640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2be006bf30 0x7f2be006e3f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.573+0000 7f2c05800640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c00150c10 0x7f2c0014bf20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.573+0000 7f2c05800640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c000cc6e0 0x7f2c0014dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.573+0000 7f2c05800640 1 --2- 172.21.15.47:0/1908691712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c000c7df0 0x7f2c001f0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.573+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 >> 172.21.15.47:0/1908691712 conn(0x7f2c0006b160 msgr2=0x7f2c00156200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:42:57.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.573+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 shutdown_connections 2024-04-24T22:42:57.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:42:57.574+0000 7f2c05800640 1 -- 172.21.15.47:0/1908691712 wait complete. 2024-04-24T22:42:57.599 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-24T22:43:00.138 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:43:00.138 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 initial writers = 330670.00 kB/sec 2024-04-24T22:43:00.138 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 initial writers = 330661.34 kB/sec 2024-04-24T22:43:00.138 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 330670.00 kB/sec 2024-04-24T22:43:00.138 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 330670.00 kB/sec 2024-04-24T22:43:00.138 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 330670.00 kB/sec 2024-04-24T22:43:00.138 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 10485760.00 kB 2024-04-24T22:43:07.600 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-24T22:43:07.600 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:43:07.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.873+0000 7fb97d400640 1 Processor -- start 2024-04-24T22:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.873+0000 7fb97d400640 1 -- start start 2024-04-24T22:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.875+0000 7fb97d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9780c2e80 0x7fb978072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.875+0000 7fb97d400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9780c8c90 con 0x7fb978152830 2024-04-24T22:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.875+0000 7fb97d400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb978152c10 con 0x7fb9780c7fc0 2024-04-24T22:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.875+0000 7fb97d400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb9780c8760 con 0x7fb9780c2e80 2024-04-24T22:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.875+0000 7fb977200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb9780c7fc0 0x7fb978162d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:49534/0 (socket says 172.21.15.47:49534) 2024-04-24T22:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.875+0000 7fb977200640 1 -- 172.21.15.47:0/3649960928 learned_addr learned my addr 172.21.15.47:0/3649960928 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.875+0000 7fb96fe00640 1 --2- 172.21.15.47:0/3649960928 >> v2:172.21.15.47:3300/0 conn(0x7fb9780c2e80 0x7fb978072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:07.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.875+0000 7fb975e00640 1 -- 172.21.15.47:0/3649960928 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3757291362 0 0) 0x7fb978152c10 con 0x7fb9780c7fc0 2024-04-24T22:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.875+0000 7fb975e00640 1 -- 172.21.15.47:0/3649960928 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb95c003520 con 0x7fb9780c7fc0 2024-04-24T22:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.876+0000 7fb96fe00640 1 -- 172.21.15.47:0/3649960928 >> v1:172.21.15.113:6789/0 conn(0x7fb9780c7fc0 legacy=0x7fb978162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.876+0000 7fb96fe00640 1 -- 172.21.15.47:0/3649960928 >> v1:172.21.15.47:6789/0 conn(0x7fb978152830 legacy=0x7fb97806d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.876+0000 7fb975e00640 1 -- 172.21.15.47:0/3649960928 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 514410729 0 0) 0x7fb9780c8c90 con 0x7fb978152830 2024-04-24T22:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.876+0000 7fb96fe00640 1 -- 172.21.15.47:0/3649960928 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9780c89b0 con 0x7fb9780c2e80 2024-04-24T22:43:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.877+0000 7fb96fe00640 1 --2- 172.21.15.47:0/3649960928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780c2e80 0x7fb978072e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb9780c88e0 tx=0x7fb96002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631219dce678fd1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.877+0000 7fb975e00640 1 -- 172.21.15.47:0/3649960928 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960037070 con 0x7fb9780c2e80 2024-04-24T22:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.877+0000 7fb975e00640 1 -- 172.21.15.47:0/3649960928 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96002acd0 con 0x7fb9780c2e80 2024-04-24T22:43:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.877+0000 7fb975e00640 1 -- 172.21.15.47:0/3649960928 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970001970 con 0x7fb9780c2e80 2024-04-24T22:43:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.878+0000 7fb97d400640 1 -- 172.21.15.47:0/3649960928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780c2e80 msgr2=0x7fb978072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.878+0000 7fb97d400640 1 --2- 172.21.15.47:0/3649960928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780c2e80 0x7fb978072e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb9780c88e0 tx=0x7fb96002a1c0 comp rx=0 tx=0).stop 2024-04-24T22:43:07.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.881+0000 7fb97d400640 1 -- 172.21.15.47:0/3649960928 shutdown_connections 2024-04-24T22:43:07.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.881+0000 7fb97d400640 1 --2- 172.21.15.47:0/3649960928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780c2e80 0x7fb978072e90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:07.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.881+0000 7fb97d400640 1 -- 172.21.15.47:0/3649960928 >> 172.21.15.47:0/3649960928 conn(0x7fb97806e730 msgr2=0x7fb97806eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:07.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.881+0000 7fb97d400640 1 -- 172.21.15.47:0/3649960928 shutdown_connections 2024-04-24T22:43:07.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.881+0000 7fb97d400640 1 -- 172.21.15.47:0/3649960928 wait complete. 2024-04-24T22:43:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.884+0000 7fb97d400640 1 Processor -- start 2024-04-24T22:43:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.884+0000 7fb97d400640 1 -- start start 2024-04-24T22:43:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.885+0000 7fb97d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780c2e80 0x7fb978151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.885+0000 7fb97d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9780c7fc0 0x7fb9781ec940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.885+0000 7fb97d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb978152830 0x7fb9780c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.885+0000 7fb97d400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb9780ca480 con 0x7fb9780c2e80 2024-04-24T22:43:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.885+0000 7fb97d400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb97806e300 con 0x7fb9780c7fc0 2024-04-24T22:43:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.885+0000 7fb97d400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb9780c4ca0 con 0x7fb978152830 2024-04-24T22:43:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.886+0000 7fb96fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9780c7fc0 0x7fb9781ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.886+0000 7fb96fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9780c7fc0 0x7fb9781ec940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:59358/0 (socket says 172.21.15.47:59358) 2024-04-24T22:43:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.886+0000 7fb96fe00640 1 -- 172.21.15.47:0/2004587100 learned_addr learned my addr 172.21.15.47:0/2004587100 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.886+0000 7fb977200640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb978152830 0x7fb9780c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.887+0000 7fb976800640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780c2e80 0x7fb978151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:07.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.887+0000 7fb96fe00640 1 -- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb978152830 msgr2=0x7fb9780c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:07.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.887+0000 7fb96fe00640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb978152830 0x7fb9780c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:07.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.887+0000 7fb96fe00640 1 -- 172.21.15.47:0/2004587100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780c2e80 msgr2=0x7fb978151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:07.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.887+0000 7fb96fe00640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780c2e80 0x7fb978151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.887+0000 7fb96fe00640 1 -- 172.21.15.47:0/2004587100 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9781ece80 con 0x7fb9780c7fc0 2024-04-24T22:43:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.887+0000 7fb976800640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780c2e80 0x7fb978151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:43:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.887+0000 7fb977200640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb978152830 0x7fb9780c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.888+0000 7fb96fe00640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9780c7fc0 0x7fb9781ec940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb960035040 tx=0x7fb9600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.888+0000 7fb96ea00640 1 -- 172.21.15.47:0/2004587100 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960037040 con 0x7fb9780c7fc0 2024-04-24T22:43:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.888+0000 7fb96ea00640 1 -- 172.21.15.47:0/2004587100 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb960004250 con 0x7fb9780c7fc0 2024-04-24T22:43:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.888+0000 7fb96ea00640 1 -- 172.21.15.47:0/2004587100 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960006e10 con 0x7fb9780c7fc0 2024-04-24T22:43:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.888+0000 7fb97d400640 1 -- 172.21.15.47:0/2004587100 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9780c6550 con 0x7fb9780c7fc0 2024-04-24T22:43:07.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.888+0000 7fb97d400640 1 -- 172.21.15.47:0/2004587100 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9781ec140 con 0x7fb9780c7fc0 2024-04-24T22:43:07.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.889+0000 7fb97d400640 1 -- 172.21.15.47:0/2004587100 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9780c6a70 con 0x7fb9780c7fc0 2024-04-24T22:43:07.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.890+0000 7fb96ea00640 1 -- 172.21.15.47:0/2004587100 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb96003a040 con 0x7fb9780c7fc0 2024-04-24T22:43:07.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.891+0000 7fb96ea00640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb95406be30 0x7fb95406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:07.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.891+0000 7fb96ea00640 1 -- 172.21.15.47:0/2004587100 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb9600ab490 con 0x7fb9780c7fc0 2024-04-24T22:43:07.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.891+0000 7fb976800640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb95406be30 0x7fb95406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:07.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.892+0000 7fb976800640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb95406be30 0x7fb95406e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb968000f60 tx=0x7fb968009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:07.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:07.897+0000 7fb96ea00640 1 -- 172.21.15.47:0/2004587100 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb9780c6a70 con 0x7fb9780c7fc0 2024-04-24T22:43:08.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.342+0000 7fb97d400640 1 -- 172.21.15.47:0/2004587100 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb978071060 con 0x7fb9780c7fc0 2024-04-24T22:43:08.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.346+0000 7fb96ea00640 1 -- 172.21.15.47:0/2004587100 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fb978071060 con 0x7fb9780c7fc0 2024-04-24T22:43:08.349 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:43:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.351+0000 7fb97d400640 1 -- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb95406be30 msgr2=0x7fb95406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.351+0000 7fb97d400640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb95406be30 0x7fb95406e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb968000f60 tx=0x7fb968009040 comp rx=0 tx=0).stop 2024-04-24T22:43:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.351+0000 7fb97d400640 1 -- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9780c7fc0 msgr2=0x7fb9781ec940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.351+0000 7fb97d400640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9780c7fc0 0x7fb9781ec940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb960035040 tx=0x7fb9600046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.352+0000 7fb97d400640 1 -- 172.21.15.47:0/2004587100 shutdown_connections 2024-04-24T22:43:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.352+0000 7fb97d400640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb95406be30 0x7fb95406e2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.352+0000 7fb97d400640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb978152830 0x7fb9780c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.352+0000 7fb97d400640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9780c7fc0 0x7fb9781ec940 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.352+0000 7fb97d400640 1 --2- 172.21.15.47:0/2004587100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9780c2e80 0x7fb978151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.352+0000 7fb97d400640 1 -- 172.21.15.47:0/2004587100 >> 172.21.15.47:0/2004587100 conn(0x7fb97806e730 msgr2=0x7fb9780cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:08.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.353+0000 7fb97d400640 1 -- 172.21.15.47:0/2004587100 shutdown_connections 2024-04-24T22:43:08.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.354+0000 7fb97d400640 1 -- 172.21.15.47:0/2004587100 wait complete. 2024-04-24T22:43:08.384 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-24T22:43:08.385 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-24T22:43:08.385 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-24T22:43:08.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:43:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.575+0000 7ff320e00640 1 Processor -- start 2024-04-24T22:43:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.575+0000 7ff320e00640 1 -- start start 2024-04-24T22:43:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.576+0000 7ff320e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff31c152960 0x7ff31c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.576+0000 7ff320e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff31c0c8fa0 0x7ff31c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.576+0000 7ff320e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff31c0c9400 con 0x7ff31c15bf60 2024-04-24T22:43:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.576+0000 7ff320e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff31c158f20 con 0x7ff31c152960 2024-04-24T22:43:08.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.576+0000 7ff320e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff31c1593b0 con 0x7ff31c0c8fa0 2024-04-24T22:43:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.577+0000 7ff319e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff31c152960 0x7ff31c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.577+0000 7ff319400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff31c0c8fa0 0x7ff31c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.577+0000 7ff319e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff31c152960 0x7ff31c0c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44370/0 (socket says 172.21.15.47:44370) 2024-04-24T22:43:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.577+0000 7ff319e00640 1 -- 172.21.15.47:0/1338748839 learned_addr learned my addr 172.21.15.47:0/1338748839 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.577+0000 7ff319e00640 1 -- 172.21.15.47:0/1338748839 >> v2:172.21.15.113:3300/0 conn(0x7ff31c0c8fa0 msgr2=0x7ff31c15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.577+0000 7ff319e00640 1 --2- 172.21.15.47:0/1338748839 >> v2:172.21.15.113:3300/0 conn(0x7ff31c0c8fa0 0x7ff31c15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.577+0000 7ff319e00640 1 -- 172.21.15.47:0/1338748839 >> v1:172.21.15.47:6789/0 conn(0x7ff31c15bf60 legacy=0x7ff31c072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.577+0000 7ff319e00640 1 -- 172.21.15.47:0/1338748839 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c0c7e50 con 0x7ff31c152960 2024-04-24T22:43:08.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.577+0000 7ff319400640 1 --2- 172.21.15.47:0/1338748839 >> v2:172.21.15.113:3300/0 conn(0x7ff31c0c8fa0 0x7ff31c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.579+0000 7ff319e00640 1 --2- 172.21.15.47:0/1338748839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c152960 0x7ff31c0c7750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff31c156080 tx=0x7ff30c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1a4d104b0d2981 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.579+0000 7ff318a00640 1 -- 172.21.15.47:0/1338748839 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c037070 con 0x7ff31c152960 2024-04-24T22:43:08.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.579+0000 7ff318a00640 1 -- 172.21.15.47:0/1338748839 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c02ae50 con 0x7ff31c152960 2024-04-24T22:43:08.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.579+0000 7ff318a00640 1 -- 172.21.15.47:0/1338748839 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c032860 con 0x7ff31c152960 2024-04-24T22:43:08.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.580+0000 7ff320e00640 1 -- 172.21.15.47:0/1338748839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c152960 msgr2=0x7ff31c0c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.580+0000 7ff320e00640 1 --2- 172.21.15.47:0/1338748839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c152960 0x7ff31c0c7750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff31c156080 tx=0x7ff30c02a300 comp rx=0 tx=0).stop 2024-04-24T22:43:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.580+0000 7ff320e00640 1 -- 172.21.15.47:0/1338748839 shutdown_connections 2024-04-24T22:43:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.580+0000 7ff320e00640 1 --2- 172.21.15.47:0/1338748839 >> v2:172.21.15.113:3300/0 conn(0x7ff31c0c8fa0 0x7ff31c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.580+0000 7ff320e00640 1 --2- 172.21.15.47:0/1338748839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c152960 0x7ff31c0c7750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.580+0000 7ff320e00640 1 -- 172.21.15.47:0/1338748839 >> 172.21.15.47:0/1338748839 conn(0x7ff31c06e580 msgr2=0x7ff31c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.581+0000 7ff320e00640 1 -- 172.21.15.47:0/1338748839 shutdown_connections 2024-04-24T22:43:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.581+0000 7ff320e00640 1 -- 172.21.15.47:0/1338748839 wait complete. 2024-04-24T22:43:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.583+0000 7ff320e00640 1 Processor -- start 2024-04-24T22:43:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.583+0000 7ff320e00640 1 -- start start 2024-04-24T22:43:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff320e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff31c0c8fa0 0x7ff31c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff320e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c152960 0x7ff31c1f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff320e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff31c15bf60 0x7ff31c1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff320e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c1588d0 con 0x7ff31c152960 2024-04-24T22:43:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff320e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c0c8130 con 0x7ff31c0c8fa0 2024-04-24T22:43:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff320e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff31c158bb0 con 0x7ff31c15bf60 2024-04-24T22:43:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff319400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c152960 0x7ff31c1f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff319400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c152960 0x7ff31c1f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44378/0 (socket says 172.21.15.47:44378) 2024-04-24T22:43:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff319400640 1 -- 172.21.15.47:0/1066744974 learned_addr learned my addr 172.21.15.47:0/1066744974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff31a800640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff31c15bf60 0x7ff31c1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.585+0000 7ff319e00640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff31c0c8fa0 0x7ff31c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.586+0000 7ff319400640 1 -- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff31c15bf60 msgr2=0x7ff31c1515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.586+0000 7ff319400640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff31c15bf60 0x7ff31c1515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.586+0000 7ff319400640 1 -- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff31c0c8fa0 msgr2=0x7ff31c15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.586+0000 7ff319400640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff31c0c8fa0 0x7ff31c15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.586+0000 7ff319400640 1 -- 172.21.15.47:0/1066744974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c1f03b0 con 0x7ff31c152960 2024-04-24T22:43:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.586+0000 7ff319400640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c152960 0x7ff31c1f4190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff30800ebe0 tx=0x7ff30800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.586+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308017070 con 0x7ff31c152960 2024-04-24T22:43:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.587+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30800cc30 con 0x7ff31c152960 2024-04-24T22:43:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.587+0000 7ff320e00640 1 -- 172.21.15.47:0/1066744974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c1f0650 con 0x7ff31c152960 2024-04-24T22:43:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.587+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308013470 con 0x7ff31c152960 2024-04-24T22:43:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.587+0000 7ff320e00640 1 -- 172.21.15.47:0/1066744974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c1f0c10 con 0x7ff31c152960 2024-04-24T22:43:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.589+0000 7ff307400640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c0c4f70 0x7ff31c1f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.589+0000 7ff307400640 1 -- 172.21.15.47:0/1066744974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c1f46d0 con 0x7ff31c0c4f70 2024-04-24T22:43:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.590+0000 7ff319e00640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c0c4f70 0x7ff31c1f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.590+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff30801a070 con 0x7ff31c152960 2024-04-24T22:43:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.592+0000 7ff312000640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff2fc06be30 0x7ff2fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.592+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff30808ade0 con 0x7ff31c152960 2024-04-24T22:43:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.592+0000 7ff31a800640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff2fc06be30 0x7ff2fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.593+0000 7ff319e00640 1 -- 172.21.15.47:0/1066744974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff31c06b3d0 con 0x7ff31c0c4f70 2024-04-24T22:43:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.593+0000 7ff31a800640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff2fc06be30 0x7ff2fc06e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3140012e0 tx=0x7ff314009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.594+0000 7ff319e00640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c0c4f70 0x7ff31c1f0e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff31c073a60 tx=0x7ff30c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.594+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c037040 con 0x7ff31c0c4f70 2024-04-24T22:43:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.595+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff30c02ae50 con 0x7ff31c0c4f70 2024-04-24T22:43:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.595+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c0c4f70 msgr2=0x7ff31c1f0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.595+0000 7ff312000640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c0c4f70 0x7ff31c1f0e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff31c073a60 tx=0x7ff30c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.595+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c0c4f70 msgr2=0x7ff31c1f0e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.595+0000 7ff312000640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c0c4f70 0x7ff31c1f0e10 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff31c073a60 tx=0x7ff30c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.602+0000 7ff307400640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c14ef20 0x7ff31c14f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.602+0000 7ff307400640 1 -- 172.21.15.47:0/1066744974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c073b90 con 0x7ff31c14ef20 2024-04-24T22:43:08.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.603+0000 7ff319e00640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c14ef20 0x7ff31c14f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.603+0000 7ff319e00640 1 -- 172.21.15.47:0/1066744974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff31c06c200 con 0x7ff31c14ef20 2024-04-24T22:43:08.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.603+0000 7ff319e00640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c14ef20 0x7ff31c14f300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff30c02a340 tx=0x7ff30c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:08.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c04d070 con 0x7ff31c14ef20 2024-04-24T22:43:08.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5041 (secure 0 0 0) 0x7ff30c04b070 con 0x7ff31c14ef20 2024-04-24T22:43:08.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c14ef20 msgr2=0x7ff31c14f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff312000640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c14ef20 0x7ff31c14f300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff30c02a340 tx=0x7ff30c008030 comp rx=0 tx=0).stop 2024-04-24T22:43:08.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff312000640 1 -- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c14ef20 msgr2=0x7ff31c14f300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:08.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff312000640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c14ef20 0x7ff31c14f300 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff30c02a340 tx=0x7ff30c008030 comp rx=0 tx=0).stop 2024-04-24T22:43:08.607 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 30, 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 128642, 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 15, 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:43:08.608 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:43:08.609 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:08.609 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:08.609 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:08.609 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:08.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:08.609 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:08.609 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:08.609 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:08.609 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:08.609 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:08.610 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:08.611 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:08.611 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:08.611 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:08.611 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.611 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:43:08.611 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:43:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:43:08.613 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:43:08.613 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:43:08.613 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:08.613 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:43:08.613 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:08.613 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:08.613 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:08.613 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:08.613 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:08.614 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:08.614 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:08.614 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:08.614 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:08.614 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:08.614 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:08.614 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.614 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:43:08.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.615 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:43:08.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.616 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:08.616 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:08.616 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:08.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:08.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:08.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:43:08.616 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:08.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:08.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:43:08.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.619 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:43:08.619 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:43:08.619 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:43:08.621 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.622 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:08.623 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:08.624 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-24T22:43:08.624 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.624 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.624 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:43:08.624 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:43:08.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff307400640 1 -- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff2fc06be30 msgr2=0x7ff2fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff307400640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff2fc06be30 0x7ff2fc06e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3140012e0 tx=0x7ff314009040 comp rx=0 tx=0).stop 2024-04-24T22:43:08.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff307400640 1 -- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c152960 msgr2=0x7ff31c1f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff307400640 1 --2- 172.21.15.47:0/1066744974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff31c152960 0x7ff31c1f4190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff30800ebe0 tx=0x7ff30800c520 comp rx=0 tx=0).stop 2024-04-24T22:43:08.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff31a800640 1 -- 172.21.15.47:0/1066744974 reap_dead start 2024-04-24T22:43:08.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff31a800640 1 -- 172.21.15.47:0/1066744974 reap_dead start 2024-04-24T22:43:08.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff307400640 1 -- 172.21.15.47:0/1066744974 shutdown_connections 2024-04-24T22:43:08.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.604+0000 7ff307400640 1 -- 172.21.15.47:0/1066744974 >> 172.21.15.47:0/1066744974 conn(0x7ff31c06e580 msgr2=0x7ff31c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:08.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.605+0000 7ff307400640 1 -- 172.21.15.47:0/1066744974 shutdown_connections 2024-04-24T22:43:08.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.605+0000 7ff307400640 1 -- 172.21.15.47:0/1066744974 wait complete. 2024-04-24T22:43:08.631 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:43:08.631 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-24T22:43:08.631 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:43:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.853+0000 7f20cc400640 1 Processor -- start 2024-04-24T22:43:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.853+0000 7f20cc400640 1 -- start start 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.854+0000 7f20cc400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20c4152960 0x7f20c40c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.854+0000 7f20cc400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.854+0000 7f20cc400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20c415bf60 0x7f20c4072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.854+0000 7f20cc400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f20c4158f20 con 0x7f20c415bf60 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.854+0000 7f20cc400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f20c41593b0 con 0x7f20c4152960 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.854+0000 7f20cc400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f20c40c8130 con 0x7f20c40c8fa0 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20c4152960 0x7f20c40c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20c4152960 0x7f20c40c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:59374/0 (socket says 172.21.15.47:59374) 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9a00640 1 -- 172.21.15.47:0/2278944190 learned_addr learned my addr 172.21.15.47:0/2278944190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c415ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45564/0 (socket says 172.21.15.47:45564) 2024-04-24T22:43:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20ca400640 1 --2- 172.21.15.47:0/2278944190 >> v2:172.21.15.47:3300/0 conn(0x7f20c415bf60 0x7f20c4072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9000640 1 -- 172.21.15.47:0/2278944190 >> v2:172.21.15.113:3300/0 conn(0x7f20c4152960 msgr2=0x7f20c40c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9000640 1 --2- 172.21.15.47:0/2278944190 >> v2:172.21.15.113:3300/0 conn(0x7f20c4152960 0x7f20c40c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9000640 1 -- 172.21.15.47:0/2278944190 >> v2:172.21.15.47:3300/0 conn(0x7f20c415bf60 msgr2=0x7f20c4072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9000640 1 --2- 172.21.15.47:0/2278944190 >> v2:172.21.15.47:3300/0 conn(0x7f20c415bf60 0x7f20c4072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9000640 1 -- 172.21.15.47:0/2278944190 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c40c9400 con 0x7f20c40c8fa0 2024-04-24T22:43:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.855+0000 7f20c9a00640 1 --2- 172.21.15.47:0/2278944190 >> v2:172.21.15.113:3300/0 conn(0x7f20c4152960 0x7f20c40c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.856+0000 7f20c9000640 1 --2- 172.21.15.47:0/2278944190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c415ba20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f20c4070600 tx=0x7f20b002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9087a51ab0b4a8f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20c3e00640 1 -- 172.21.15.47:0/2278944190 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0037070 con 0x7f20c40c8fa0 2024-04-24T22:43:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20c3e00640 1 -- 172.21.15.47:0/2278944190 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b0032440 con 0x7f20c40c8fa0 2024-04-24T22:43:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20c3e00640 1 -- 172.21.15.47:0/2278944190 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0032810 con 0x7f20c40c8fa0 2024-04-24T22:43:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20cc400640 1 -- 172.21.15.47:0/2278944190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 msgr2=0x7f20c415ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20cc400640 1 --2- 172.21.15.47:0/2278944190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c415ba20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f20c4070600 tx=0x7f20b002a1a0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20cc400640 1 -- 172.21.15.47:0/2278944190 shutdown_connections 2024-04-24T22:43:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20cc400640 1 --2- 172.21.15.47:0/2278944190 >> v2:172.21.15.47:3300/0 conn(0x7f20c415bf60 0x7f20c4072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20cc400640 1 --2- 172.21.15.47:0/2278944190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c415ba20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20cc400640 1 --2- 172.21.15.47:0/2278944190 >> v2:172.21.15.113:3300/0 conn(0x7f20c4152960 0x7f20c40c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20cc400640 1 -- 172.21.15.47:0/2278944190 >> 172.21.15.47:0/2278944190 conn(0x7f20c406e580 msgr2=0x7f20c406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20cc400640 1 -- 172.21.15.47:0/2278944190 shutdown_connections 2024-04-24T22:43:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.857+0000 7f20cc400640 1 -- 172.21.15.47:0/2278944190 wait complete. 2024-04-24T22:43:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.859+0000 7f20cc400640 1 Processor -- start 2024-04-24T22:43:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.859+0000 7f20cc400640 1 -- start start 2024-04-24T22:43:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20cc400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c4072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20cc400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20c4152960 0x7f20c40c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20cc400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c415bf60 0x7f20c41595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20cc400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41588d0 con 0x7f20c4152960 2024-04-24T22:43:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20cc400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41585f0 con 0x7f20c415bf60 2024-04-24T22:43:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20cc400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f20c4158bb0 con 0x7f20c40c8fa0 2024-04-24T22:43:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20c9a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c4072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20c9a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c4072320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45576/0 (socket says 172.21.15.47:45576) 2024-04-24T22:43:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20c9a00640 1 -- 172.21.15.47:0/796661694 learned_addr learned my addr 172.21.15.47:0/796661694 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20ca400640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c415bf60 0x7f20c41595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.860+0000 7f20c9000640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20c4152960 0x7f20c40c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20c9a00640 1 -- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c415bf60 msgr2=0x7f20c41595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20c9a00640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c415bf60 0x7f20c41595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20c9a00640 1 -- 172.21.15.47:0/796661694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20c4152960 msgr2=0x7f20c40c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20c9a00640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20c4152960 0x7f20c40c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20c9a00640 1 -- 172.21.15.47:0/796661694 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c40729b0 con 0x7f20c40c8fa0 2024-04-24T22:43:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20c9a00640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c4072320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e9b0 tx=0x7f20b800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b800ca00 con 0x7f20c40c8fa0 2024-04-24T22:43:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b800cba0 con 0x7f20c40c8fa0 2024-04-24T22:43:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8004030 con 0x7f20c40c8fa0 2024-04-24T22:43:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20cc400640 1 -- 172.21.15.47:0/796661694 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c40c5420 con 0x7f20c40c8fa0 2024-04-24T22:43:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.861+0000 7f20cc400640 1 -- 172.21.15.47:0/796661694 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c40c59e0 con 0x7f20c40c8fa0 2024-04-24T22:43:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.862+0000 7f20cc400640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c4159b30 0x7f20c41f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.862+0000 7f20cc400640 1 -- 172.21.15.47:0/796661694 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f20c40c5dc0 con 0x7f20c4159b30 2024-04-24T22:43:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.863+0000 7f20c9000640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c4159b30 0x7f20c41f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.864+0000 7f20c9000640 1 -- 172.21.15.47:0/796661694 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20c4073b90 con 0x7f20c4159b30 2024-04-24T22:43:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.864+0000 7f20c9000640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c4159b30 0x7f20c41f3740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f20c415c3f0 tx=0x7f20b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.865+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0043070 con 0x7f20c4159b30 2024-04-24T22:43:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.865+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20b80041d0 con 0x7f20c40c8fa0 2024-04-24T22:43:08.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.866+0000 7f20c1600640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f20a406bd60 0x7f20a406e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.866+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f20b808b2a0 con 0x7f20c40c8fa0 2024-04-24T22:43:08.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.866+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f20b00093c0 con 0x7f20c4159b30 2024-04-24T22:43:08.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.866+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c4159b30 msgr2=0x7f20c41f3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.866+0000 7f20c1600640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c4159b30 0x7f20c41f3740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f20c415c3f0 tx=0x7f20b00046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.866+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c4159b30 msgr2=0x7f20c41f3740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:08.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.866+0000 7f20c1600640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c4159b30 0x7f20c41f3740 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f20c415c3f0 tx=0x7f20b00046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:08.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.866+0000 7f20ca400640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f20a406bd60 0x7f20a406e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.870+0000 7f20ca400640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f20a406bd60 0x7f20a406e220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f20c40c51e0 tx=0x7f20bc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:08.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.881+0000 7f20cc400640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c41f3c80 0x7f20c4146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:08.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.881+0000 7f20cc400640 1 -- 172.21.15.47:0/796661694 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f20c406c200 con 0x7f20c41f3c80 2024-04-24T22:43:08.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.881+0000 7f20c9000640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c41f3c80 0x7f20c4146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:08.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.882+0000 7f20c9000640 1 -- 172.21.15.47:0/796661694 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f20c4151d20 con 0x7f20c41f3c80 2024-04-24T22:43:08.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.882+0000 7f20c9000640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c41f3c80 0x7f20c4146130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f20b002a1e0 tx=0x7f20b0008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:08.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.882+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0037040 con 0x7f20c41f3c80 2024-04-24T22:43:08.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.883+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5066 (secure 0 0 0) 0x7f20b003c640 con 0x7f20c41f3c80 2024-04-24T22:43:08.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.883+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c41f3c80 msgr2=0x7f20c4146130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.883+0000 7f20c1600640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c41f3c80 0x7f20c4146130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f20b002a1e0 tx=0x7f20b0008030 comp rx=0 tx=0).stop 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.883+0000 7f20c1600640 1 -- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c41f3c80 msgr2=0x7f20c4146130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.883+0000 7f20c1600640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20c41f3c80 0x7f20c4146130 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f20b002a1e0 tx=0x7f20b0008030 comp rx=0 tx=0).stop 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 30, 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 230452, 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:43:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:43:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:43:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 15, 2024-04-24T22:43:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:43:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:43:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.890 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:43:08.890 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:43:08.890 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:43:08.890 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:43:08.890 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:08.891 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:43:08.892 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:08.892 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:08.892 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:08.893 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:43:08.894 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.895 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:08.897 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:43:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.899 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:08.900 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:08.901 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7 2024-04-24T22:43:08.901 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.901 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.901 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:08.902 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:08.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:08.903 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:43:08.903 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:43:08.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.884+0000 7f20cc400640 1 -- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f20a406bd60 msgr2=0x7f20a406e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.884+0000 7f20cc400640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f20a406bd60 0x7f20a406e220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f20c40c51e0 tx=0x7f20bc00b040 comp rx=0 tx=0).stop 2024-04-24T22:43:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.884+0000 7f20cc400640 1 -- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 msgr2=0x7f20c4072320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.884+0000 7f20cc400640 1 --2- 172.21.15.47:0/796661694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20c40c8fa0 0x7f20c4072320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e9b0 tx=0x7f20b800ee70 comp rx=0 tx=0).stop 2024-04-24T22:43:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.884+0000 7f20ca400640 1 -- 172.21.15.47:0/796661694 reap_dead start 2024-04-24T22:43:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.884+0000 7f20ca400640 1 -- 172.21.15.47:0/796661694 reap_dead start 2024-04-24T22:43:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.884+0000 7f20cc400640 1 -- 172.21.15.47:0/796661694 shutdown_connections 2024-04-24T22:43:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.884+0000 7f20cc400640 1 -- 172.21.15.47:0/796661694 >> 172.21.15.47:0/796661694 conn(0x7f20c406e580 msgr2=0x7f20c4156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.884+0000 7f20cc400640 1 -- 172.21.15.47:0/796661694 shutdown_connections 2024-04-24T22:43:08.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:08.884+0000 7f20cc400640 1 -- 172.21.15.47:0/796661694 wait complete. 2024-04-24T22:43:08.907 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:43:08.907 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-24T22:43:08.907 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-24T22:43:09.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.203+0000 7f5731200640 1 Processor -- start 2024-04-24T22:43:09.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.203+0000 7f5731200640 1 -- start start 2024-04-24T22:43:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.204+0000 7f5731200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c2850 0x7f572c06e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.204+0000 7f5731200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f572c0c6880 0x7f572c169160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.204+0000 7f5731200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f572c162d40 con 0x7f572c06e1b0 2024-04-24T22:43:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.204+0000 7f5731200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f572c161e70 con 0x7f572c0c6880 2024-04-24T22:43:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.204+0000 7f5731200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f572c0c71f0 con 0x7f572c0c2850 2024-04-24T22:43:09.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.205+0000 7f5723e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c2850 0x7f572c06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.205+0000 7f572b400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f572c06e1b0 0x7f572c162630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:49538/0 (socket says 172.21.15.47:49538) 2024-04-24T22:43:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.205+0000 7f5723e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c2850 0x7f572c06e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45592/0 (socket says 172.21.15.47:45592) 2024-04-24T22:43:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.205+0000 7f572b400640 1 -- 172.21.15.47:0/4152442182 learned_addr learned my addr 172.21.15.47:0/4152442182 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:43:09.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.205+0000 7f572be00640 1 --2- 172.21.15.47:0/4152442182 >> v2:172.21.15.47:3300/0 conn(0x7f572c0c6880 0x7f572c169160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.205+0000 7f5723e00640 1 -- 172.21.15.47:0/4152442182 >> v2:172.21.15.47:3300/0 conn(0x7f572c0c6880 msgr2=0x7f572c169160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.205+0000 7f5723e00640 1 --2- 172.21.15.47:0/4152442182 >> v2:172.21.15.47:3300/0 conn(0x7f572c0c6880 0x7f572c169160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.205+0000 7f5723e00640 1 -- 172.21.15.47:0/4152442182 >> v1:172.21.15.113:6789/0 conn(0x7f572c06e1b0 legacy=0x7f572c162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.205+0000 7f5723e00640 1 -- 172.21.15.47:0/4152442182 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572c0c7810 con 0x7f572c0c2850 2024-04-24T22:43:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.206+0000 7f5723e00640 1 --2- 172.21.15.47:0/4152442182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c2850 0x7f572c06e590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f57140097a0 tx=0x7f571402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe49c89ec30131ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.207+0000 7f572aa00640 1 -- 172.21.15.47:0/4152442182 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714037070 con 0x7f572c0c2850 2024-04-24T22:43:09.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.207+0000 7f572aa00640 1 -- 172.21.15.47:0/4152442182 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5714032440 con 0x7f572c0c2850 2024-04-24T22:43:09.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.207+0000 7f572aa00640 1 -- 172.21.15.47:0/4152442182 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714032810 con 0x7f572c0c2850 2024-04-24T22:43:09.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.207+0000 7f5731200640 1 -- 172.21.15.47:0/4152442182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c2850 msgr2=0x7f572c06e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.207+0000 7f5731200640 1 --2- 172.21.15.47:0/4152442182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c2850 0x7f572c06e590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f57140097a0 tx=0x7f571402a230 comp rx=0 tx=0).stop 2024-04-24T22:43:09.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.207+0000 7f5731200640 1 -- 172.21.15.47:0/4152442182 shutdown_connections 2024-04-24T22:43:09.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.207+0000 7f5731200640 1 --2- 172.21.15.47:0/4152442182 >> v2:172.21.15.47:3300/0 conn(0x7f572c0c6880 0x7f572c169160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.207+0000 7f5731200640 1 --2- 172.21.15.47:0/4152442182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c2850 0x7f572c06e590 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.208+0000 7f5731200640 1 -- 172.21.15.47:0/4152442182 >> 172.21.15.47:0/4152442182 conn(0x7f572c0cb900 msgr2=0x7f572c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:09.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.208+0000 7f5731200640 1 -- 172.21.15.47:0/4152442182 shutdown_connections 2024-04-24T22:43:09.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.208+0000 7f5731200640 1 -- 172.21.15.47:0/4152442182 wait complete. 2024-04-24T22:43:09.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.209+0000 7f5731200640 1 Processor -- start 2024-04-24T22:43:09.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.209+0000 7f5731200640 1 -- start start 2024-04-24T22:43:09.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.210+0000 7f5731200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f572c06e1b0 0x7f572c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.210+0000 7f5731200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0c2850 0x7f572c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.210+0000 7f5731200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c6880 0x7f572c14ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.210+0000 7f5731200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f572c15fcd0 con 0x7f572c0c2850 2024-04-24T22:43:09.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.210+0000 7f5731200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f572c0c7dd0 con 0x7f572c06e1b0 2024-04-24T22:43:09.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.210+0000 7f5731200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f572c158e00 con 0x7f572c0c6880 2024-04-24T22:43:09.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f572be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c6880 0x7f572c14ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f572be00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c6880 0x7f572c14ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45596/0 (socket says 172.21.15.47:45596) 2024-04-24T22:43:09.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f572be00640 1 -- 172.21.15.47:0/1678289031 learned_addr learned my addr 172.21.15.47:0/1678289031 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:09.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f5723e00640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0c2850 0x7f572c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f572b400640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f572c06e1b0 0x7f572c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f572be00640 1 -- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f572c06e1b0 msgr2=0x7f572c15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f572be00640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f572c06e1b0 0x7f572c15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f572be00640 1 -- 172.21.15.47:0/1678289031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0c2850 msgr2=0x7f572c15e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f572be00640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0c2850 0x7f572c15e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f572be00640 1 -- 172.21.15.47:0/1678289031 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572c15e9e0 con 0x7f572c0c6880 2024-04-24T22:43:09.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.211+0000 7f5723e00640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f572c0c2850 0x7f572c15e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:09.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.212+0000 7f572be00640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c6880 0x7f572c14ed80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5724009d30 tx=0x7f572400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:09.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.212+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724018020 con 0x7f572c0c6880 2024-04-24T22:43:09.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.212+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5724014480 con 0x7f572c0c6880 2024-04-24T22:43:09.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.212+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724014870 con 0x7f572c0c6880 2024-04-24T22:43:09.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.212+0000 7f5731200640 1 -- 172.21.15.47:0/1678289031 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572c0c46a0 con 0x7f572c0c6880 2024-04-24T22:43:09.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.213+0000 7f5731200640 1 -- 172.21.15.47:0/1678289031 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f572c15d9c0 con 0x7f572c0c6880 2024-04-24T22:43:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.214+0000 7f5731200640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15d460 0x7f572c14f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.214+0000 7f5731200640 1 -- 172.21.15.47:0/1678289031 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f572c068d50 con 0x7f572c15d460 2024-04-24T22:43:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.214+0000 7f572b400640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15d460 0x7f572c14f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.214+0000 7f572b400640 1 -- 172.21.15.47:0/1678289031 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f572c0c2420 con 0x7f572c15d460 2024-04-24T22:43:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.215+0000 7f572b400640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15d460 0x7f572c14f2c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f571c000f60 tx=0x7f571c00e940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:09.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.215+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c005ac0 con 0x7f572c15d460 2024-04-24T22:43:09.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.215+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f571c018070 con 0x7f572c15d460 2024-04-24T22:43:09.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.215+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15d460 msgr2=0x7f572c14f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.215+0000 7f5723400640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15d460 0x7f572c14f2c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f571c000f60 tx=0x7f571c00e940 comp rx=0 tx=0).stop 2024-04-24T22:43:09.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.215+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15d460 msgr2=0x7f572c14f2c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:09.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.215+0000 7f5723400640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15d460 0x7f572c14f2c0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f571c000f60 tx=0x7f571c00e940 comp rx=0 tx=0).stop 2024-04-24T22:43:09.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.216+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5724005d10 con 0x7f572c0c6880 2024-04-24T22:43:09.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.217+0000 7f5723400640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f570806b910 0x7f570806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.217+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f572408b9e0 con 0x7f572c0c6880 2024-04-24T22:43:09.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.217+0000 7f572b400640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f570806b910 0x7f570806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.219+0000 7f572b400640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f570806b910 0x7f570806ddd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f571c015040 tx=0x7f571c012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:09.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.229+0000 7f5731200640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15dc00 0x7f572c1f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.229+0000 7f5731200640 1 -- 172.21.15.47:0/1678289031 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f572c04f5e0 con 0x7f572c15dc00 2024-04-24T22:43:09.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.229+0000 7f5723e00640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15dc00 0x7f572c1f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.229+0000 7f5723e00640 1 -- 172.21.15.47:0/1678289031 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f572c0c7450 con 0x7f572c15dc00 2024-04-24T22:43:09.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.230+0000 7f5723e00640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15dc00 0x7f572c1f7d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f572c0c6810 tx=0x7f57140046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:09.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.230+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714043070 con 0x7f572c15dc00 2024-04-24T22:43:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.230+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4555 (secure 0 0 0) 0x7f5714032440 con 0x7f572c15dc00 2024-04-24T22:43:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.231+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15dc00 msgr2=0x7f572c1f7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.231+0000 7f5723400640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15dc00 0x7f572c1f7d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f572c0c6810 tx=0x7f57140046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.231+0000 7f5723400640 1 -- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15dc00 msgr2=0x7f572c1f7d90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:09.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.231+0000 7f5723400640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c15dc00 0x7f572c1f7d90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.234 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:43:09.234 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:09.234 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:09.234 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:43:09.234 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 30, 2024-04-24T22:43:09.234 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 22873, 2024-04-24T22:43:09.234 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:43:09.234 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:43:09.234 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 16, 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:09.235 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:09.236 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:43:09.237 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:09.238 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:09.239 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:09.240 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:43:09.241 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:09.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:09.243 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:09.244 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-24T22:43:09.245 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:09.246 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:09.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:09.246 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:43:09.246 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:43:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.231+0000 7f5731200640 1 -- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f570806b910 msgr2=0x7f570806ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.231+0000 7f5731200640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f570806b910 0x7f570806ddd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f571c015040 tx=0x7f571c012000 comp rx=0 tx=0).stop 2024-04-24T22:43:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.231+0000 7f5731200640 1 -- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c6880 msgr2=0x7f572c14ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.231+0000 7f5731200640 1 --2- 172.21.15.47:0/1678289031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f572c0c6880 0x7f572c14ed80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5724009d30 tx=0x7f572400c490 comp rx=0 tx=0).stop 2024-04-24T22:43:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.231+0000 7f572be00640 1 -- 172.21.15.47:0/1678289031 reap_dead start 2024-04-24T22:43:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.231+0000 7f572be00640 1 -- 172.21.15.47:0/1678289031 reap_dead start 2024-04-24T22:43:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.232+0000 7f5731200640 1 -- 172.21.15.47:0/1678289031 shutdown_connections 2024-04-24T22:43:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.232+0000 7f5731200640 1 -- 172.21.15.47:0/1678289031 >> 172.21.15.47:0/1678289031 conn(0x7f572c0cb900 msgr2=0x7f572c162630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:09.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.232+0000 7f5731200640 1 -- 172.21.15.47:0/1678289031 shutdown_connections 2024-04-24T22:43:09.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.232+0000 7f5731200640 1 -- 172.21.15.47:0/1678289031 wait complete. 2024-04-24T22:43:09.251 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:43:09.251 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-04-24T22:43:09.251 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-24T22:43:09.251 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-04-24T22:43:09.252 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-24T22:43:09.252 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-24T22:43:09.252 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:43:09.268 INFO:tasks.ceph.mon.b:Stopped 2024-04-24T22:43:09.269 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-24T22:43:09.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:43:09.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.446+0000 7fbe39200640 1 Processor -- start 2024-04-24T22:43:09.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.446+0000 7fbe39200640 1 -- start start 2024-04-24T22:43:09.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.447+0000 7fbe39200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158fc0 0x7fbe340c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.447+0000 7fbe39200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe340c1280 0x7fbe340c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.447+0000 7fbe39200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe341621d0 con 0x7fbe34158be0 2024-04-24T22:43:09.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.447+0000 7fbe39200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbe341624b0 con 0x7fbe340c1280 2024-04-24T22:43:09.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.447+0000 7fbe39200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbe34162790 con 0x7fbe34158fc0 2024-04-24T22:43:09.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.447+0000 7fbe31e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbe34158be0 legacy=0x7fbe340c78f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-04-24T22:43:09.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.448+0000 7fbe29400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158fc0 0x7fbe340c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.448+0000 7fbe29400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158fc0 0x7fbe340c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45670/0 (socket says 172.21.15.47:45670) 2024-04-24T22:43:09.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.448+0000 7fbe29400640 1 -- 172.21.15.47:0/2229579649 learned_addr learned my addr 172.21.15.47:0/2229579649 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:09.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.448+0000 7fbe32800640 1 --2- 172.21.15.47:0/2229579649 >> v2:172.21.15.47:3300/0 conn(0x7fbe340c1280 0x7fbe340c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.448+0000 7fbe29400640 1 -- 172.21.15.47:0/2229579649 >> v2:172.21.15.47:3300/0 conn(0x7fbe340c1280 msgr2=0x7fbe340c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.448+0000 7fbe29400640 1 --2- 172.21.15.47:0/2229579649 >> v2:172.21.15.47:3300/0 conn(0x7fbe340c1280 0x7fbe340c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.448+0000 7fbe29400640 1 -- 172.21.15.47:0/2229579649 >> v1:172.21.15.113:6789/0 conn(0x7fbe34158be0 legacy=0x7fbe340c78f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.448+0000 7fbe29400640 1 -- 172.21.15.47:0/2229579649 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe34162a70 con 0x7fbe34158fc0 2024-04-24T22:43:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.450+0000 7fbe29400640 1 --2- 172.21.15.47:0/2229579649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158fc0 0x7fbe340c8590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009e50 tx=0x7fbe1c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=457002ece3e12baf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.450+0000 7fbe31400640 1 -- 172.21.15.47:0/2229579649 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c036070 con 0x7fbe34158fc0 2024-04-24T22:43:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.450+0000 7fbe31400640 1 -- 172.21.15.47:0/2229579649 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1c031600 con 0x7fbe34158fc0 2024-04-24T22:43:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.450+0000 7fbe31400640 1 -- 172.21.15.47:0/2229579649 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c0319d0 con 0x7fbe34158fc0 2024-04-24T22:43:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.451+0000 7fbe39200640 1 -- 172.21.15.47:0/2229579649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158fc0 msgr2=0x7fbe340c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.451+0000 7fbe39200640 1 --2- 172.21.15.47:0/2229579649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158fc0 0x7fbe340c8590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009e50 tx=0x7fbe1c02a6d0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.451+0000 7fbe39200640 1 -- 172.21.15.47:0/2229579649 shutdown_connections 2024-04-24T22:43:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.451+0000 7fbe39200640 1 --2- 172.21.15.47:0/2229579649 >> v2:172.21.15.47:3300/0 conn(0x7fbe340c1280 0x7fbe340c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.451+0000 7fbe39200640 1 --2- 172.21.15.47:0/2229579649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158fc0 0x7fbe340c8590 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.451+0000 7fbe39200640 1 -- 172.21.15.47:0/2229579649 >> 172.21.15.47:0/2229579649 conn(0x7fbe3406e580 msgr2=0x7fbe3406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.451+0000 7fbe39200640 1 -- 172.21.15.47:0/2229579649 shutdown_connections 2024-04-24T22:43:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.451+0000 7fbe39200640 1 -- 172.21.15.47:0/2229579649 wait complete. 2024-04-24T22:43:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.454+0000 7fbe39200640 1 Processor -- start 2024-04-24T22:43:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.454+0000 7fbe39200640 1 -- start start 2024-04-24T22:43:09.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.455+0000 7fbe39200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe340c1280 0x7fbe340c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.455+0000 7fbe39200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158be0 0x7fbe341f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.455+0000 7fbe39200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe34158fc0 0x7fbe34152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.456+0000 7fbe39200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3415ed50 con 0x7fbe340c1280 2024-04-24T22:43:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.456+0000 7fbe39200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3415ebd0 con 0x7fbe34158fc0 2024-04-24T22:43:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.456+0000 7fbe39200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3415eed0 con 0x7fbe34158be0 2024-04-24T22:43:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.456+0000 7fbe32800640 1 -- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe34158fc0 msgr2=0x7fbe34152ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3300/0 2024-04-24T22:43:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.456+0000 7fbe32800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe34158fc0 0x7fbe34152ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.456+0000 7fbe29400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158be0 0x7fbe341f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.456+0000 7fbe29400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158be0 0x7fbe341f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45680/0 (socket says 172.21.15.47:45680) 2024-04-24T22:43:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.456+0000 7fbe29400640 1 -- 172.21.15.47:0/1092000077 learned_addr learned my addr 172.21.15.47:0/1092000077 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.457+0000 7fbe31e00640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe340c1280 0x7fbe340c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.457+0000 7fbe29400640 1 -- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe34158fc0 msgr2=0x7fbe34152ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:43:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.457+0000 7fbe29400640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe34158fc0 0x7fbe34152ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.457+0000 7fbe29400640 1 -- 172.21.15.47:0/1092000077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe340c1280 msgr2=0x7fbe340c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.457+0000 7fbe29400640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe340c1280 0x7fbe340c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.457+0000 7fbe29400640 1 -- 172.21.15.47:0/1092000077 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe341f7e60 con 0x7fbe34158be0 2024-04-24T22:43:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.458+0000 7fbe29400640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158be0 0x7fbe341f4c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009f70 tx=0x7fbe1c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.459+0000 7fbe2aa00640 1 -- 172.21.15.47:0/1092000077 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c03f020 con 0x7fbe34158be0 2024-04-24T22:43:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.459+0000 7fbe2aa00640 1 -- 172.21.15.47:0/1092000077 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1c037040 con 0x7fbe34158be0 2024-04-24T22:43:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.459+0000 7fbe2aa00640 1 -- 172.21.15.47:0/1092000077 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c036070 con 0x7fbe34158be0 2024-04-24T22:43:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.459+0000 7fbe39200640 1 -- 172.21.15.47:0/1092000077 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe341f8120 con 0x7fbe34158be0 2024-04-24T22:43:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.459+0000 7fbe39200640 1 -- 172.21.15.47:0/1092000077 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe341f86e0 con 0x7fbe34158be0 2024-04-24T22:43:09.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.460+0000 7fbe39200640 1 -- 172.21.15.47:0/1092000077 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe341f44d0 con 0x7fbe34158be0 2024-04-24T22:43:09.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.462+0000 7fbe2aa00640 1 -- 172.21.15.47:0/1092000077 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe1c042020 con 0x7fbe34158be0 2024-04-24T22:43:09.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.463+0000 7fbe2aa00640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbe1006bde0 0x7fbe1006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:09.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.463+0000 7fbe2aa00640 1 -- 172.21.15.47:0/1092000077 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbe1c0ab3c0 con 0x7fbe34158be0 2024-04-24T22:43:09.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.464+0000 7fbe31e00640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbe1006bde0 0x7fbe1006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:09.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.464+0000 7fbe31e00640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbe1006bde0 0x7fbe1006e2a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009370 tx=0x7fbe24009300 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.468+0000 7fbe2aa00640 1 -- 172.21.15.47:0/1092000077 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbe1c049030 con 0x7fbe34158be0 2024-04-24T22:43:09.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.921+0000 7fbe39200640 1 -- 172.21.15.47:0/1092000077 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbe34073b90 con 0x7fbe34158be0 2024-04-24T22:43:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.922+0000 7fbe2aa00640 1 -- 172.21.15.47:0/1092000077 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fbe1c079ce0 con 0x7fbe34158be0 2024-04-24T22:43:09.925 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:43:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.926+0000 7fbe39200640 1 -- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbe1006bde0 msgr2=0x7fbe1006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.926+0000 7fbe39200640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbe1006bde0 0x7fbe1006e2a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009370 tx=0x7fbe24009300 comp rx=0 tx=0).stop 2024-04-24T22:43:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.926+0000 7fbe39200640 1 -- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158be0 msgr2=0x7fbe341f4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.926+0000 7fbe39200640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158be0 0x7fbe341f4c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009f70 tx=0x7fbe1c004a40 comp rx=0 tx=0).stop 2024-04-24T22:43:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.929+0000 7fbe39200640 1 -- 172.21.15.47:0/1092000077 shutdown_connections 2024-04-24T22:43:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.929+0000 7fbe39200640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbe1006bde0 0x7fbe1006e2a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.929+0000 7fbe39200640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe34158fc0 0x7fbe34152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.929+0000 7fbe39200640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe34158be0 0x7fbe341f4c00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.929+0000 7fbe39200640 1 --2- 172.21.15.47:0/1092000077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe340c1280 0x7fbe340c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.929+0000 7fbe39200640 1 -- 172.21.15.47:0/1092000077 >> 172.21.15.47:0/1092000077 conn(0x7fbe3406e580 msgr2=0x7fbe34156de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.929+0000 7fbe39200640 1 -- 172.21.15.47:0/1092000077 shutdown_connections 2024-04-24T22:43:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:09.930+0000 7fbe39200640 1 -- 172.21.15.47:0/1092000077 wait complete. 2024-04-24T22:43:10.906 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:43:10.914 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:43:12.959 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:43:13.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.190+0000 7fb3fe000640 1 Processor -- start 2024-04-24T22:43:13.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.190+0000 7fb3fe000640 1 -- start start 2024-04-24T22:43:13.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.192+0000 7fb3fe000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f8073670 0x7fb3f81576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:13.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.192+0000 7fb3fe000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3f815aa30 con 0x7fb3f80c1a00 2024-04-24T22:43:13.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.192+0000 7fb3fe000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3f815ad10 con 0x7fb3f8151780 2024-04-24T22:43:13.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.192+0000 7fb3fe000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f815aff0 con 0x7fb3f8073670 2024-04-24T22:43:13.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.193+0000 7fb3f7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f8073670 0x7fb3f81576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:13.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.193+0000 7fb3f7400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fb3f8151780 legacy=0x7fb3f80c8590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-04-24T22:43:13.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.193+0000 7fb3f6a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb3f80c1a00 0x7fb3f8072f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53350/0 (socket says 172.21.15.47:53350) 2024-04-24T22:43:13.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.193+0000 7fb3f6a00640 1 -- 172.21.15.47:0/900781357 learned_addr learned my addr 172.21.15.47:0/900781357 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:43:13.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.193+0000 7fb3f7e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f8073670 0x7fb3f81576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45688/0 (socket says 172.21.15.47:45688) 2024-04-24T22:43:13.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.193+0000 7fb3f7e00640 1 -- 172.21.15.47:0/900781357 >> v1:172.21.15.113:6789/0 conn(0x7fb3f8151780 legacy=0x7fb3f80c8590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:13.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.193+0000 7fb3f7e00640 1 -- 172.21.15.47:0/900781357 >> v1:172.21.15.47:6789/0 conn(0x7fb3f80c1a00 legacy=0x7fb3f8072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:13.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.193+0000 7fb3f7e00640 1 -- 172.21.15.47:0/900781357 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f815b2d0 con 0x7fb3f8073670 2024-04-24T22:43:13.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.193+0000 7fb3f6000640 1 -- 172.21.15.47:0/900781357 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1341938279 0 0) 0x7fb3f815aa30 con 0x7fb3f80c1a00 2024-04-24T22:43:13.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.195+0000 7fb3f7e00640 1 --2- 172.21.15.47:0/900781357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f8073670 0x7fb3f81576f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009dc0 tx=0x7fb3f002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbffe1b7487bfa2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:13.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.196+0000 7fb3f6000640 1 -- 172.21.15.47:0/900781357 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0007740 con 0x7fb3f8073670 2024-04-24T22:43:13.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.196+0000 7fb3f6000640 1 -- 172.21.15.47:0/900781357 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f00078e0 con 0x7fb3f8073670 2024-04-24T22:43:13.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.196+0000 7fb3f6000640 1 -- 172.21.15.47:0/900781357 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0007cb0 con 0x7fb3f8073670 2024-04-24T22:43:13.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.197+0000 7fb3fe000640 1 -- 172.21.15.47:0/900781357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f8073670 msgr2=0x7fb3f81576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:13.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.197+0000 7fb3fe000640 1 --2- 172.21.15.47:0/900781357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f8073670 0x7fb3f81576f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009dc0 tx=0x7fb3f002a710 comp rx=0 tx=0).stop 2024-04-24T22:43:13.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.197+0000 7fb3fe000640 1 -- 172.21.15.47:0/900781357 shutdown_connections 2024-04-24T22:43:13.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.197+0000 7fb3fe000640 1 --2- 172.21.15.47:0/900781357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f8073670 0x7fb3f81576f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:13.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.197+0000 7fb3fe000640 1 -- 172.21.15.47:0/900781357 >> 172.21.15.47:0/900781357 conn(0x7fb3f806e580 msgr2=0x7fb3f806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:13.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.197+0000 7fb3fe000640 1 -- 172.21.15.47:0/900781357 shutdown_connections 2024-04-24T22:43:13.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.197+0000 7fb3fe000640 1 -- 172.21.15.47:0/900781357 wait complete. 2024-04-24T22:43:13.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.200+0000 7fb3fe000640 1 Processor -- start 2024-04-24T22:43:13.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.200+0000 7fb3fe000640 1 -- start start 2024-04-24T22:43:13.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.201+0000 7fb3fe000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8073670 0x7fb3f80c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:13.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.201+0000 7fb3fe000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f80c1a00 0x7fb3f8159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:13.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.201+0000 7fb3fe000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f8151780 0x7fb3f814f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:13.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.201+0000 7fb3fe000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f80c8e20 con 0x7fb3f8151780 2024-04-24T22:43:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.201+0000 7fb3fe000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f80c8ca0 con 0x7fb3f8073670 2024-04-24T22:43:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.201+0000 7fb3fe000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f8157c30 con 0x7fb3f80c1a00 2024-04-24T22:43:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.201+0000 7fb3f7400640 1 -- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8073670 msgr2=0x7fb3f80c9900 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3300/0 2024-04-24T22:43:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.202+0000 7fb3f7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f8151780 0x7fb3f814f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.202+0000 7fb3f7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8073670 0x7fb3f80c9900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.202+0000 7fb3f7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f8151780 0x7fb3f814f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44516/0 (socket says 172.21.15.47:44516) 2024-04-24T22:43:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.202+0000 7fb3f7e00640 1 -- 172.21.15.47:0/1514486309 learned_addr learned my addr 172.21.15.47:0/1514486309 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.202+0000 7fb3f6a00640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f80c1a00 0x7fb3f8159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.202+0000 7fb3f7e00640 1 -- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f80c1a00 msgr2=0x7fb3f8159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.202+0000 7fb3f7e00640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f80c1a00 0x7fb3f8159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:13.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.202+0000 7fb3f7e00640 1 -- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8073670 msgr2=0x7fb3f80c9900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:43:13.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.202+0000 7fb3f7e00640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8073670 0x7fb3f80c9900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:13.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.202+0000 7fb3f7e00640 1 -- 172.21.15.47:0/1514486309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f8150e60 con 0x7fb3f8151780 2024-04-24T22:43:13.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.203+0000 7fb3f7e00640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f8151780 0x7fb3f814f8b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009ee0 tx=0x7fb3f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:13.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.203+0000 7fb3ef400640 1 -- 172.21.15.47:0/1514486309 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0041030 con 0x7fb3f8151780 2024-04-24T22:43:13.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.203+0000 7fb3fe000640 1 -- 172.21.15.47:0/1514486309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f813b990 con 0x7fb3f8151780 2024-04-24T22:43:13.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.203+0000 7fb3fe000640 1 -- 172.21.15.47:0/1514486309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f813bf50 con 0x7fb3f8151780 2024-04-24T22:43:13.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.203+0000 7fb3ef400640 1 -- 172.21.15.47:0/1514486309 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f00319c0 con 0x7fb3f8151780 2024-04-24T22:43:13.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.204+0000 7fb3ef400640 1 -- 172.21.15.47:0/1514486309 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0041030 con 0x7fb3f8151780 2024-04-24T22:43:13.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.205+0000 7fb3fe000640 1 -- 172.21.15.47:0/1514486309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d0005290 con 0x7fb3f8151780 2024-04-24T22:43:13.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.207+0000 7fb3ef400640 1 -- 172.21.15.47:0/1514486309 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3f0044030 con 0x7fb3f8151780 2024-04-24T22:43:13.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.209+0000 7fb3ef400640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb3d806c010 0x7fb3d806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:13.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.209+0000 7fb3ef400640 1 -- 172.21.15.47:0/1514486309 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb3f00ac1c0 con 0x7fb3f8151780 2024-04-24T22:43:13.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.209+0000 7fb3f7400640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb3d806c010 0x7fb3d806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:13.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.210+0000 7fb3f7400640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb3d806c010 0x7fb3d806e4d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8003930 tx=0x7fb3e8006d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:13.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.211+0000 7fb3ef400640 1 -- 172.21.15.47:0/1514486309 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb3f007ab40 con 0x7fb3f8151780 2024-04-24T22:43:13.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.714+0000 7fb3fe000640 1 -- 172.21.15.47:0/1514486309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb3d0005580 con 0x7fb3f8151780 2024-04-24T22:43:13.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.716+0000 7fb3ef400640 1 -- 172.21.15.47:0/1514486309 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fb3f007a440 con 0x7fb3f8151780 2024-04-24T22:43:13.719 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:43:13.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.720+0000 7fb3fe000640 1 -- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb3d806c010 msgr2=0x7fb3d806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:13.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.720+0000 7fb3fe000640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb3d806c010 0x7fb3d806e4d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8003930 tx=0x7fb3e8006d30 comp rx=0 tx=0).stop 2024-04-24T22:43:13.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.720+0000 7fb3fe000640 1 -- 172.21.15.47:0/1514486309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f8151780 msgr2=0x7fb3f814f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:13.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.720+0000 7fb3fe000640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f8151780 0x7fb3f814f8b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009ee0 tx=0x7fb3f0004a40 comp rx=0 tx=0).stop 2024-04-24T22:43:13.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.722+0000 7fb3fe000640 1 -- 172.21.15.47:0/1514486309 shutdown_connections 2024-04-24T22:43:13.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.722+0000 7fb3fe000640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb3d806c010 0x7fb3d806e4d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:13.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.722+0000 7fb3fe000640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f8151780 0x7fb3f814f8b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:13.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.722+0000 7fb3fe000640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3f80c1a00 0x7fb3f8159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:13.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.722+0000 7fb3fe000640 1 --2- 172.21.15.47:0/1514486309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f8073670 0x7fb3f80c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:13.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.722+0000 7fb3fe000640 1 -- 172.21.15.47:0/1514486309 >> 172.21.15.47:0/1514486309 conn(0x7fb3f806e580 msgr2=0x7fb3f8061b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:13.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.725+0000 7fb3fe000640 1 -- 172.21.15.47:0/1514486309 shutdown_connections 2024-04-24T22:43:13.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:13.725+0000 7fb3fe000640 1 -- 172.21.15.47:0/1514486309 wait complete. 2024-04-24T22:43:16.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:43:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.027+0000 7fc0cd600640 1 Processor -- start 2024-04-24T22:43:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.028+0000 7fc0cd600640 1 -- start start 2024-04-24T22:43:17.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.030+0000 7fc0cd600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0c80c7df0 0x7fc0c8150c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:17.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.030+0000 7fc0cd600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0c80c8360 0x7fc0c8062410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:17.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.030+0000 7fc0cd600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c8156d00 con 0x7fc0c80cc1a0 2024-04-24T22:43:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.030+0000 7fc0cd600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c8151150 con 0x7fc0c80c7df0 2024-04-24T22:43:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.030+0000 7fc0cd600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c8156fe0 con 0x7fc0c80c8360 2024-04-24T22:43:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.030+0000 7fc0c7400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc0c80cc1a0 0x7fc0c80720c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34848/0 (socket says 172.21.15.47:34848) 2024-04-24T22:43:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.030+0000 7fc0c6000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc0c80c8360 msgr2=0x7fc0c8062410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-04-24T22:43:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.030+0000 7fc0c6000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0c80c8360 0x7fc0c8062410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.030+0000 7fc0c6a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0c80c7df0 0x7fc0c8150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.030+0000 7fc0c7400640 1 -- 172.21.15.47:0/2438630581 learned_addr learned my addr 172.21.15.47:0/2438630581 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:43:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.031+0000 7fc0c6a00640 1 -- 172.21.15.47:0/2438630581 >> v2:172.21.15.113:3300/0 conn(0x7fc0c80c8360 msgr2=0x7fc0c8062410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.031+0000 7fc0c6a00640 1 --2- 172.21.15.47:0/2438630581 >> v2:172.21.15.113:3300/0 conn(0x7fc0c80c8360 0x7fc0c8062410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.031+0000 7fc0c6a00640 1 -- 172.21.15.47:0/2438630581 >> v1:172.21.15.47:6789/0 conn(0x7fc0c80cc1a0 legacy=0x7fc0c80720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.031+0000 7fc0c6a00640 1 -- 172.21.15.47:0/2438630581 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c81572c0 con 0x7fc0c80c7df0 2024-04-24T22:43:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.032+0000 7fc0c6a00640 1 --2- 172.21.15.47:0/2438630581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0c80c7df0 0x7fc0c8150c10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8073660 tx=0x7fc0b802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486e34325491fd58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.033+0000 7fc0c5600640 1 -- 172.21.15.47:0/2438630581 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8037070 con 0x7fc0c80c7df0 2024-04-24T22:43:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.033+0000 7fc0c5600640 1 -- 172.21.15.47:0/2438630581 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b8032440 con 0x7fc0c80c7df0 2024-04-24T22:43:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.033+0000 7fc0c5600640 1 -- 172.21.15.47:0/2438630581 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8032810 con 0x7fc0c80c7df0 2024-04-24T22:43:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.034+0000 7fc0cd600640 1 -- 172.21.15.47:0/2438630581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0c80c7df0 msgr2=0x7fc0c8150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.034+0000 7fc0cd600640 1 --2- 172.21.15.47:0/2438630581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0c80c7df0 0x7fc0c8150c10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8073660 tx=0x7fc0b802a250 comp rx=0 tx=0).stop 2024-04-24T22:43:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.034+0000 7fc0cd600640 1 -- 172.21.15.47:0/2438630581 shutdown_connections 2024-04-24T22:43:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.034+0000 7fc0cd600640 1 --2- 172.21.15.47:0/2438630581 >> v2:172.21.15.113:3300/0 conn(0x7fc0c80c8360 0x7fc0c8062410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.034+0000 7fc0cd600640 1 --2- 172.21.15.47:0/2438630581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0c80c7df0 0x7fc0c8150c10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.034+0000 7fc0cd600640 1 -- 172.21.15.47:0/2438630581 >> 172.21.15.47:0/2438630581 conn(0x7fc0c806b160 msgr2=0x7fc0c806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.035+0000 7fc0cd600640 1 -- 172.21.15.47:0/2438630581 shutdown_connections 2024-04-24T22:43:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.035+0000 7fc0cd600640 1 -- 172.21.15.47:0/2438630581 wait complete. 2024-04-24T22:43:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.036+0000 7fc0cd600640 1 Processor -- start 2024-04-24T22:43:17.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.037+0000 7fc0cd600640 1 -- start start 2024-04-24T22:43:17.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.037+0000 7fc0cd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c80c7df0 0x7fc0c8061df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:17.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.037+0000 7fc0cd600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c80c8360 0x7fc0c81f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:17.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.037+0000 7fc0cd600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0c80cc1a0 0x7fc0c813c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:17.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.037+0000 7fc0cd600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c8071ba0 con 0x7fc0c80cc1a0 2024-04-24T22:43:17.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.037+0000 7fc0cd600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c8157880 con 0x7fc0c80c7df0 2024-04-24T22:43:17.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.037+0000 7fc0cd600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c81569d0 con 0x7fc0c80c8360 2024-04-24T22:43:17.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.038+0000 7fc0c6a00640 1 -- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c80c7df0 msgr2=0x7fc0c8061df0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3300/0 2024-04-24T22:43:17.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.038+0000 7fc0c6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c80c7df0 0x7fc0c8061df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:17.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.038+0000 7fc0c7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0c80cc1a0 0x7fc0c813c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:17.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.038+0000 7fc0c6000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c80c8360 0x7fc0c81f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:17.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.038+0000 7fc0c6000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c80c8360 0x7fc0c81f3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45862/0 (socket says 172.21.15.47:45862) 2024-04-24T22:43:17.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.038+0000 7fc0c6000640 1 -- 172.21.15.47:0/1153383931 learned_addr learned my addr 172.21.15.47:0/1153383931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:17.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0c6000640 1 -- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c80c7df0 msgr2=0x7fc0c8061df0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:43:17.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0c6000640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c80c7df0 0x7fc0c8061df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:17.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0c6000640 1 -- 172.21.15.47:0/1153383931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0c80cc1a0 msgr2=0x7fc0c813c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:17.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0c6000640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0c80cc1a0 0x7fc0c813c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:17.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0c6000640 1 -- 172.21.15.47:0/1153383931 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c8153f80 con 0x7fc0c80c8360 2024-04-24T22:43:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0c7400640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0c80cc1a0 0x7fc0c813c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0c6000640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c80c8360 0x7fc0c81f3b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400b580 tx=0x7fc0b400ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0bea00640 1 -- 172.21.15.47:0/1153383931 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4013020 con 0x7fc0c80c8360 2024-04-24T22:43:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0bea00640 1 -- 172.21.15.47:0/1153383931 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b4003de0 con 0x7fc0c80c8360 2024-04-24T22:43:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0bea00640 1 -- 172.21.15.47:0/1153383931 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b40052b0 con 0x7fc0c80c8360 2024-04-24T22:43:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.039+0000 7fc0cd600640 1 -- 172.21.15.47:0/1153383931 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c8154220 con 0x7fc0c80c8360 2024-04-24T22:43:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.040+0000 7fc0cd600640 1 -- 172.21.15.47:0/1153383931 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c813f9b0 con 0x7fc0c80c8360 2024-04-24T22:43:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.041+0000 7fc0cd600640 1 -- 172.21.15.47:0/1153383931 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c8073b90 con 0x7fc0c80c8360 2024-04-24T22:43:17.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.042+0000 7fc0bea00640 1 -- 172.21.15.47:0/1153383931 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0b4016020 con 0x7fc0c80c8360 2024-04-24T22:43:17.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.043+0000 7fc0bea00640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc0a806bde0 0x7fc0a806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:17.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.043+0000 7fc0bea00640 1 -- 172.21.15.47:0/1153383931 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc0b408ac00 con 0x7fc0c80c8360 2024-04-24T22:43:17.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.043+0000 7fc0c6a00640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc0a806bde0 0x7fc0a806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:17.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.044+0000 7fc0c6a00640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc0a806bde0 0x7fc0a806e2a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8034000 tx=0x7fc0b802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.048+0000 7fc0bea00640 1 -- 172.21.15.47:0/1153383931 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0b401d020 con 0x7fc0c80c8360 2024-04-24T22:43:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.436+0000 7fc0cd600640 1 -- 172.21.15.47:0/1153383931 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc0c8150c10 con 0x7fc0c80c8360 2024-04-24T22:43:17.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.437+0000 7fc0bea00640 1 -- 172.21.15.47:0/1153383931 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fc0b4059520 con 0x7fc0c80c8360 2024-04-24T22:43:17.440 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:43:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.439+0000 7fc0cd600640 1 -- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc0a806bde0 msgr2=0x7fc0a806e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.439+0000 7fc0cd600640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc0a806bde0 0x7fc0a806e2a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8034000 tx=0x7fc0b802e000 comp rx=0 tx=0).stop 2024-04-24T22:43:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.439+0000 7fc0cd600640 1 -- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c80c8360 msgr2=0x7fc0c81f3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.439+0000 7fc0cd600640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c80c8360 0x7fc0c81f3b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400b580 tx=0x7fc0b400ba40 comp rx=0 tx=0).stop 2024-04-24T22:43:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.442+0000 7fc0cd600640 1 -- 172.21.15.47:0/1153383931 shutdown_connections 2024-04-24T22:43:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.442+0000 7fc0cd600640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc0a806bde0 0x7fc0a806e2a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.442+0000 7fc0cd600640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0c80cc1a0 0x7fc0c813c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:17.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.442+0000 7fc0cd600640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0c80c8360 0x7fc0c81f3b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.442+0000 7fc0cd600640 1 --2- 172.21.15.47:0/1153383931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0c80c7df0 0x7fc0c8061df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.442+0000 7fc0cd600640 1 -- 172.21.15.47:0/1153383931 >> 172.21.15.47:0/1153383931 conn(0x7fc0c806b160 msgr2=0x7fc0c8072930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.442+0000 7fc0cd600640 1 -- 172.21.15.47:0/1153383931 shutdown_connections 2024-04-24T22:43:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:17.442+0000 7fc0cd600640 1 -- 172.21.15.47:0/1153383931 wait complete. 2024-04-24T22:43:20.463 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:43:20.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.713+0000 7ff703e00640 1 Processor -- start 2024-04-24T22:43:20.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.713+0000 7ff703e00640 1 -- start start 2024-04-24T22:43:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.714+0000 7ff703e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.714+0000 7ff703e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.714+0000 7ff703e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6fc0c9400 con 0x7ff6fc15bf60 2024-04-24T22:43:20.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.714+0000 7ff703e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff6fc158f20 con 0x7ff6fc152960 2024-04-24T22:43:20.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.714+0000 7ff703e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff6fc1593b0 con 0x7ff6fc0c8fa0 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff700a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff701400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff700a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45868/0 (socket says 172.21.15.47:45868) 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff700a00640 1 -- 172.21.15.47:0/1581388869 learned_addr learned my addr 172.21.15.47:0/1581388869 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff700a00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 msgr2=0x7ff6fc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff700a00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 msgr2=0x7ff6fc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff701400640 1 -- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 msgr2=0x7ff6fc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff701400640 1 -- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 msgr2=0x7ff6fc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.715+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:20.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.916+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:20.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.916+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:20.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.916+0000 7ff701400640 1 -- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 msgr2=0x7ff6fc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:43:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.916+0000 7ff701400640 1 -- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 msgr2=0x7ff6fc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.916+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.917+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:43:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.917+0000 7ff700a00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 msgr2=0x7ff6fc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:43:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.917+0000 7ff700a00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 msgr2=0x7ff6fc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.917+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:20.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:20.917+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T22:43:21.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:21.318+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:21.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:21.318+0000 7ff700a00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 msgr2=0x7ff6fc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:43:21.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:21.318+0000 7ff700a00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 msgr2=0x7ff6fc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:21.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:21.318+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:21.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:21.318+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:43:21.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:21.318+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:21.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:21.319+0000 7ff701400640 1 -- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 msgr2=0x7ff6fc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:43:21.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:21.319+0000 7ff701400640 1 -- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 msgr2=0x7ff6fc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:21.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:21.319+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:21.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:21.319+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T22:43:22.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:22.119+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:22.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:22.120+0000 7ff700a00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 msgr2=0x7ff6fc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:43:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:22.120+0000 7ff700a00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 msgr2=0x7ff6fc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:22.120+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:22.120+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:43:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:22.120+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:22.120+0000 7ff701400640 1 -- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 msgr2=0x7ff6fc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:43:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:22.120+0000 7ff701400640 1 -- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 msgr2=0x7ff6fc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:22.120+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:22.120+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T22:43:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.441+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 416786917 0 0) 0x7ff6fc0c9400 con 0x7ff6fc15bf60 2024-04-24T22:43:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.441+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6e0003520 con 0x7ff6fc15bf60 2024-04-24T22:43:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1190162223 0 0) 0x7ff6e0003520 con 0x7ff6fc15bf60 2024-04-24T22:43:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 msgr2=0x7ff6fc15ba20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc0c8fa0 0x7ff6fc15ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 msgr2=0x7ff6fc0c7750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 --2- 172.21.15.47:0/1581388869 >> v2:172.21.15.47:3300/0 conn(0x7ff6fc152960 0x7ff6fc0c7750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6fc0c7e50 con 0x7ff6fc15bf60 2024-04-24T22:43:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 567141988 0 0) 0x7ff6ec001670 con 0x7ff6fc15bf60 2024-04-24T22:43:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 >> v1:172.21.15.47:6789/0 conn(0x7ff6fc15bf60 legacy=0x7ff6fc072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e00039b0 0x7ff6e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e0004800 0x7ff6e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.445+0000 7ff6fbe00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6e00051c0 0x7ff6e00141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:23.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.446+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc158f20 con 0x7ff6e00051c0 2024-04-24T22:43:23.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.446+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1593b0 con 0x7ff6e0004800 2024-04-24T22:43:23.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.446+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e00147f0 con 0x7ff6e00039b0 2024-04-24T22:43:23.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.447+0000 7ff700a00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e0004800 msgr2=0x7ff6e0004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-04-24T22:43:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.448+0000 7ff700a00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e0004800 0x7ff6e0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.448+0000 7ff701400640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e00039b0 0x7ff6e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.448+0000 7ff701e00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6e00051c0 0x7ff6e00141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.450+0000 7ff701e00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e00039b0 msgr2=0x7ff6e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.450+0000 7ff701e00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6e00039b0 0x7ff6e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.450+0000 7ff701e00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e0004800 msgr2=0x7ff6e0004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.450+0000 7ff701e00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e0004800 0x7ff6e0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:23.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.450+0000 7ff701e00640 1 -- 172.21.15.47:0/1581388869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6fc0c7e50 con 0x7ff6e00051c0 2024-04-24T22:43:23.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.451+0000 7ff701e00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6e00051c0 0x7ff6e00141d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec007fd0 tx=0x7ff6e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e75c5661225752 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:23.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.453+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec036070 con 0x7ff6e00051c0 2024-04-24T22:43:23.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.453+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ec007250 con 0x7ff6e00051c0 2024-04-24T22:43:23.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.453+0000 7ff6fbe00640 1 -- 172.21.15.47:0/1581388869 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec007ca0 con 0x7ff6e00051c0 2024-04-24T22:43:23.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.454+0000 7ff703e00640 1 -- 172.21.15.47:0/1581388869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6e00051c0 msgr2=0x7ff6e00141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:23.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.454+0000 7ff703e00640 1 --2- 172.21.15.47:0/1581388869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6e00051c0 0x7ff6e00141d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec007fd0 tx=0x7ff6e0005830 comp rx=0 tx=0).stop 2024-04-24T22:43:23.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.455+0000 7ff701e00640 1 -- 172.21.15.47:0/1581388869 reap_dead start 2024-04-24T22:43:23.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.455+0000 7ff703e00640 1 -- 172.21.15.47:0/1581388869 shutdown_connections 2024-04-24T22:43:23.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.455+0000 7ff703e00640 1 -- 172.21.15.47:0/1581388869 >> 172.21.15.47:0/1581388869 conn(0x7ff6fc06e580 msgr2=0x7ff6fc06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:23.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.460+0000 7ff703e00640 1 -- 172.21.15.47:0/1581388869 shutdown_connections 2024-04-24T22:43:23.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.460+0000 7ff703e00640 1 -- 172.21.15.47:0/1581388869 wait complete. 2024-04-24T22:43:23.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.463+0000 7ff703e00640 1 Processor -- start 2024-04-24T22:43:23.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.463+0000 7ff703e00640 1 -- start start 2024-04-24T22:43:23.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.466+0000 7ff703e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6fc0c3ba0 0x7ff6fc15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:23.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.466+0000 7ff703e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6fc0c4d80 0x7ff6fc14b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.466+0000 7ff703e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc1ec690 0x7ff6fc0c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.466+0000 7ff703e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1588d0 con 0x7ff6fc0c4d80 2024-04-24T22:43:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.466+0000 7ff703e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc0c8130 con 0x7ff6fc0c3ba0 2024-04-24T22:43:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.466+0000 7ff703e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff6fc158bb0 con 0x7ff6fc1ec690 2024-04-24T22:43:23.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.467+0000 7ff701e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc1ec690 0x7ff6fc0c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:23.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.467+0000 7ff701e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc1ec690 0x7ff6fc0c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45920/0 (socket says 172.21.15.47:45920) 2024-04-24T22:43:23.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.467+0000 7ff701e00640 1 -- 172.21.15.47:0/2406854302 learned_addr learned my addr 172.21.15.47:0/2406854302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:23.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.467+0000 7ff701400640 1 -- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6fc0c3ba0 msgr2=0x7ff6fc15b240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3300/0 2024-04-24T22:43:23.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.467+0000 7ff701400640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6fc0c3ba0 0x7ff6fc15b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:23.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.468+0000 7ff701e00640 1 -- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6fc0c3ba0 msgr2=0x7ff6fc15b240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:43:23.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.468+0000 7ff701e00640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6fc0c3ba0 0x7ff6fc15b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:23.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.468+0000 7ff701e00640 1 -- 172.21.15.47:0/2406854302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6fc0c4d80 msgr2=0x7ff6fc14b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:23.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.468+0000 7ff701e00640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6fc0c4d80 0x7ff6fc14b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:23.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.468+0000 7ff701e00640 1 -- 172.21.15.47:0/2406854302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6fc0c44c0 con 0x7ff6fc1ec690 2024-04-24T22:43:23.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.469+0000 7ff701e00640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc1ec690 0x7ff6fc0c3f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00c2a0 tx=0x7ff6ec001970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:23.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.470+0000 7ff6f9600640 1 -- 172.21.15.47:0/2406854302 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec041070 con 0x7ff6fc1ec690 2024-04-24T22:43:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.470+0000 7ff6f9600640 1 -- 172.21.15.47:0/2406854302 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ec007400 con 0x7ff6fc1ec690 2024-04-24T22:43:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.470+0000 7ff6f9600640 1 -- 172.21.15.47:0/2406854302 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec036040 con 0x7ff6fc1ec690 2024-04-24T22:43:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.470+0000 7ff703e00640 1 -- 172.21.15.47:0/2406854302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6fc0c5210 con 0x7ff6fc1ec690 2024-04-24T22:43:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.470+0000 7ff703e00640 1 -- 172.21.15.47:0/2406854302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6fc15a760 con 0x7ff6fc1ec690 2024-04-24T22:43:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.474+0000 7ff703e00640 1 -- 172.21.15.47:0/2406854302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d4005290 con 0x7ff6fc1ec690 2024-04-24T22:43:23.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.479+0000 7ff6f9600640 1 -- 172.21.15.47:0/2406854302 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6ec044070 con 0x7ff6fc1ec690 2024-04-24T22:43:23.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.480+0000 7ff6f9600640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff6dc06be60 0x7ff6dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:23.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.481+0000 7ff701400640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff6dc06be60 0x7ff6dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:23.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.481+0000 7ff6f9600640 1 -- 172.21.15.47:0/2406854302 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff6ec0a9db0 con 0x7ff6fc1ec690 2024-04-24T22:43:23.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.486+0000 7ff701400640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff6dc06be60 0x7ff6dc06e320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400da20 tx=0x7ff6f400d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:23.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.487+0000 7ff6f9600640 1 -- 172.21.15.47:0/2406854302 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff6ec078100 con 0x7ff6fc1ec690 2024-04-24T22:43:23.940 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":34,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:43:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.931+0000 7ff703e00640 1 -- 172.21.15.47:0/2406854302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff6d4005580 con 0x7ff6fc1ec690 2024-04-24T22:43:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.936+0000 7ff6f9600640 1 -- 172.21.15.47:0/2406854302 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7ff6ec07c400 con 0x7ff6fc1ec690 2024-04-24T22:43:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.944+0000 7ff703e00640 1 -- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff6dc06be60 msgr2=0x7ff6dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.944+0000 7ff703e00640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff6dc06be60 0x7ff6dc06e320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400da20 tx=0x7ff6f400d040 comp rx=0 tx=0).stop 2024-04-24T22:43:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.944+0000 7ff703e00640 1 -- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc1ec690 msgr2=0x7ff6fc0c3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.944+0000 7ff703e00640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc1ec690 0x7ff6fc0c3f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00c2a0 tx=0x7ff6ec001970 comp rx=0 tx=0).stop 2024-04-24T22:43:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.944+0000 7ff703e00640 1 -- 172.21.15.47:0/2406854302 shutdown_connections 2024-04-24T22:43:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.944+0000 7ff703e00640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff6dc06be60 0x7ff6dc06e320 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.944+0000 7ff703e00640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6fc1ec690 0x7ff6fc0c3f80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.944+0000 7ff703e00640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6fc0c4d80 0x7ff6fc14b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.944+0000 7ff703e00640 1 --2- 172.21.15.47:0/2406854302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6fc0c3ba0 0x7ff6fc15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.944+0000 7ff703e00640 1 -- 172.21.15.47:0/2406854302 >> 172.21.15.47:0/2406854302 conn(0x7ff6fc06e580 msgr2=0x7ff6fc157b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.945+0000 7ff703e00640 1 -- 172.21.15.47:0/2406854302 shutdown_connections 2024-04-24T22:43:23.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:23.946+0000 7ff703e00640 1 -- 172.21.15.47:0/2406854302 wait complete. 2024-04-24T22:43:23.978 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-24T22:43:23.978 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-24T22:43:23.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:43:24.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.271+0000 7f83c0600640 1 Processor -- start 2024-04-24T22:43:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.272+0000 7f83c0600640 1 -- start start 2024-04-24T22:43:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.273+0000 7f83c0600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83b8158be0 0x7f83b80c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.273+0000 7f83c0600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83b8158fc0 0x7f83b80c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.273+0000 7f83c0600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b81621d0 con 0x7f83b80c1280 2024-04-24T22:43:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.273+0000 7f83c0600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f83b81624b0 con 0x7f83b8158be0 2024-04-24T22:43:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.273+0000 7f83c0600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f83b8162790 con 0x7f83b8158fc0 2024-04-24T22:43:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.273+0000 7f83bd200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83b8158fc0 msgr2=0x7f83b80c8590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-04-24T22:43:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.273+0000 7f83bd200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83b8158fc0 0x7f83b80c8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.273+0000 7f83be600640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f83b80c1280 0x7f83b815ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34862/0 (socket says 172.21.15.47:34862) 2024-04-24T22:43:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.273+0000 7f83be600640 1 -- 172.21.15.47:0/36719394 learned_addr learned my addr 172.21.15.47:0/36719394 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:43:24.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.273+0000 7f83bdc00640 1 --2- 172.21.15.47:0/36719394 >> v2:172.21.15.47:3300/0 conn(0x7f83b8158be0 0x7f83b80c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.274+0000 7f83bdc00640 1 -- 172.21.15.47:0/36719394 >> v2:172.21.15.113:3300/0 conn(0x7f83b8158fc0 msgr2=0x7f83b80c8590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:24.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.274+0000 7f83bdc00640 1 --2- 172.21.15.47:0/36719394 >> v2:172.21.15.113:3300/0 conn(0x7f83b8158fc0 0x7f83b80c8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.274+0000 7f83bdc00640 1 -- 172.21.15.47:0/36719394 >> v1:172.21.15.47:6789/0 conn(0x7f83b80c1280 legacy=0x7f83b815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.274+0000 7f83bdc00640 1 -- 172.21.15.47:0/36719394 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b8162a70 con 0x7f83b8158be0 2024-04-24T22:43:24.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.276+0000 7f83bdc00640 1 --2- 172.21.15.47:0/36719394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b8158be0 0x7f83b80c78f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83b80c76a0 tx=0x7f83ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5231fde35760292c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:24.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.276+0000 7f83b7e00640 1 -- 172.21.15.47:0/36719394 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac037070 con 0x7f83b8158be0 2024-04-24T22:43:24.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.276+0000 7f83b7e00640 1 -- 172.21.15.47:0/36719394 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ac032440 con 0x7f83b8158be0 2024-04-24T22:43:24.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.276+0000 7f83b7e00640 1 -- 172.21.15.47:0/36719394 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac032810 con 0x7f83b8158be0 2024-04-24T22:43:24.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.277+0000 7f83c0600640 1 -- 172.21.15.47:0/36719394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b8158be0 msgr2=0x7f83b80c78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.277+0000 7f83c0600640 1 --2- 172.21.15.47:0/36719394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b8158be0 0x7f83b80c78f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83b80c76a0 tx=0x7f83ac02a270 comp rx=0 tx=0).stop 2024-04-24T22:43:24.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.278+0000 7f83c0600640 1 -- 172.21.15.47:0/36719394 shutdown_connections 2024-04-24T22:43:24.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.278+0000 7f83c0600640 1 --2- 172.21.15.47:0/36719394 >> v2:172.21.15.113:3300/0 conn(0x7f83b8158fc0 0x7f83b80c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.278+0000 7f83c0600640 1 --2- 172.21.15.47:0/36719394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b8158be0 0x7f83b80c78f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.278+0000 7f83c0600640 1 -- 172.21.15.47:0/36719394 >> 172.21.15.47:0/36719394 conn(0x7f83b806e580 msgr2=0x7f83b806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:24.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.279+0000 7f83c0600640 1 -- 172.21.15.47:0/36719394 shutdown_connections 2024-04-24T22:43:24.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.279+0000 7f83c0600640 1 -- 172.21.15.47:0/36719394 wait complete. 2024-04-24T22:43:24.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.280+0000 7f83c0600640 1 Processor -- start 2024-04-24T22:43:24.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.281+0000 7f83c0600640 1 -- start start 2024-04-24T22:43:24.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.282+0000 7f83c0600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b80c1280 0x7f83b80cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.282+0000 7f83c0600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83b8158be0 0x7f83b80c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.282+0000 7f83c0600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b8158fc0 0x7f83b8144280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.282+0000 7f83bd200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83b8158be0 0x7f83b80c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.282+0000 7f83bd200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83b8158be0 0x7f83b80c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45936/0 (socket says 172.21.15.47:45936) 2024-04-24T22:43:24.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.283+0000 7f83be600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b8158fc0 0x7f83b8144280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.283+0000 7f83be600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b8158fc0 0x7f83b8144280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34900/0 (socket says 172.21.15.47:34900) 2024-04-24T22:43:24.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.283+0000 7f83bdc00640 1 -- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b80c1280 msgr2=0x7f83b80cc200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3300/0 2024-04-24T22:43:24.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.283+0000 7f83bd200640 1 -- 172.21.15.47:0/3723153550 learned_addr learned my addr 172.21.15.47:0/3723153550 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:24.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.283+0000 7f83bdc00640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b80c1280 0x7f83b80cc200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:24.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.283+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f83b80c1050 con 0x7f83b8158fc0 2024-04-24T22:43:24.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.283+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f83b80c0ed0 con 0x7f83b80c1280 2024-04-24T22:43:24.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.283+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f83b815f2e0 con 0x7f83b8158be0 2024-04-24T22:43:24.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.284+0000 7f83bd200640 1 -- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b80c1280 msgr2=0x7f83b80cc200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:43:24.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.284+0000 7f83bd200640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b80c1280 0x7f83b80cc200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.284+0000 7f83bd200640 1 -- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b8158fc0 msgr2=0x7f83b8144280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.284+0000 7f83bd200640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b8158fc0 0x7f83b8144280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.284+0000 7f83bd200640 1 -- 172.21.15.47:0/3723153550 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b8142920 con 0x7f83b8158be0 2024-04-24T22:43:24.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.284+0000 7f83be600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b8158fc0 0x7f83b8144280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:24.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.284+0000 7f83bd200640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83b8158be0 0x7f83b80c53c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f83a400b530 tx=0x7f83a400b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:24.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.285+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4013020 con 0x7f83b8158be0 2024-04-24T22:43:24.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.285+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a4003dc0 con 0x7f83b8158be0 2024-04-24T22:43:24.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.285+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4005290 con 0x7f83b8158be0 2024-04-24T22:43:24.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.287+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b8142640 con 0x7f83b8158be0 2024-04-24T22:43:24.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.287+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b80c9cf0 con 0x7f83b8158be0 2024-04-24T22:43:24.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.292+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83a4016020 con 0x7f83b8158be0 2024-04-24T22:43:24.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.292+0000 7f83c0600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b815e4b0 0x7f83b81447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.292+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f83b80cc740 con 0x7f83b815e4b0 2024-04-24T22:43:24.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.294+0000 7f83b5600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f839806bde0 0x7f839806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.294+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f83a408aa10 con 0x7f83b8158be0 2024-04-24T22:43:24.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.295+0000 7f83bdc00640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b815e4b0 0x7f83b81447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.297+0000 7f83bdc00640 1 -- 172.21.15.47:0/3723153550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83b81423f0 con 0x7f83b815e4b0 2024-04-24T22:43:24.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.298+0000 7f83be600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f839806bde0 0x7f839806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.299+0000 7f83be600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f839806bde0 0x7f839806e2a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005c00 tx=0x7f83b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:24.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.299+0000 7f83bdc00640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b815e4b0 0x7f83b81447c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f83b80b2180 tx=0x7f83ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:24.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.300+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac042070 con 0x7f83b815e4b0 2024-04-24T22:43:24.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.301+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f83ac03d440 con 0x7f83b815e4b0 2024-04-24T22:43:24.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.301+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b815e4b0 msgr2=0x7f83b81447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.301+0000 7f83b5600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b815e4b0 0x7f83b81447c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f83b80b2180 tx=0x7f83ac0046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.301+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b815e4b0 msgr2=0x7f83b81447c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:24.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.301+0000 7f83b5600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b815e4b0 0x7f83b81447c0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f83b80b2180 tx=0x7f83ac0046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.325+0000 7f83c0600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b80ca060 0x7f83b815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.325+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f83b80c5900 con 0x7f83b80ca060 2024-04-24T22:43:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.326+0000 7f83bdc00640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b80ca060 0x7f83b815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.327+0000 7f83bdc00640 1 -- 172.21.15.47:0/3723153550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f83b8073b90 con 0x7f83b80ca060 2024-04-24T22:43:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.328+0000 7f83bdc00640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b80ca060 0x7f83b815bec0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000c00 tx=0x7f83ac009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.328+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac042070 con 0x7f83b80ca060 2024-04-24T22:43:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.329+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5199 (secure 0 0 0) 0x7f83ac03d5f0 con 0x7f83b80ca060 2024-04-24T22:43:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.329+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b80ca060 msgr2=0x7f83b815bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.329+0000 7f83b5600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b80ca060 0x7f83b815bec0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000c00 tx=0x7f83ac009190 comp rx=0 tx=0).stop 2024-04-24T22:43:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.329+0000 7f83b5600640 1 -- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b80ca060 msgr2=0x7f83b815bec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.329+0000 7f83b5600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83b80ca060 0x7f83b815bec0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000c00 tx=0x7f83ac009190 comp rx=0 tx=0).stop 2024-04-24T22:43:24.332 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:43:24.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:24.332 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:24.332 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:43:24.332 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 34, 2024-04-24T22:43:24.332 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 144365, 2024-04-24T22:43:24.332 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:43:24.332 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:43:24.332 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:43:24.332 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:24.333 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:24.334 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:24.335 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:24.335 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.335 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:43:24.335 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:43:24.335 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:43:24.336 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:24.337 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:43:24.338 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:24.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:43:24.340 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:24.341 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:24.343 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.344 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:43:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:43:24.346 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:43:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.329+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f839806bde0 msgr2=0x7f839806e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.329+0000 7f83c0600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f839806bde0 0x7f839806e2a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005c00 tx=0x7f83b0009040 comp rx=0 tx=0).stop 2024-04-24T22:43:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.329+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83b8158be0 msgr2=0x7f83b80c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.329+0000 7f83c0600640 1 --2- 172.21.15.47:0/3723153550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83b8158be0 0x7f83b80c53c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f83a400b530 tx=0x7f83a400b9f0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.329+0000 7f83be600640 1 -- 172.21.15.47:0/3723153550 reap_dead start 2024-04-24T22:43:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.330+0000 7f83be600640 1 -- 172.21.15.47:0/3723153550 reap_dead start 2024-04-24T22:43:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.330+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 shutdown_connections 2024-04-24T22:43:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.330+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 >> 172.21.15.47:0/3723153550 conn(0x7f83b806e580 msgr2=0x7f83b80c16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:24.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.330+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 shutdown_connections 2024-04-24T22:43:24.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.330+0000 7f83c0600640 1 -- 172.21.15.47:0/3723153550 wait complete. 2024-04-24T22:43:24.352 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-24T22:43:24.352 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-24T22:43:24.352 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-24T22:43:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.566+0000 7f6ceb200640 1 Processor -- start 2024-04-24T22:43:24.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.566+0000 7f6ceb200640 1 -- start start 2024-04-24T22:43:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.568+0000 7f6ceb200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 0x7f6cec0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.568+0000 7f6ceb200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cec0cbe00 con 0x7f6cec0cb970 2024-04-24T22:43:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.568+0000 7f6ceb200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cec15b2c0 con 0x7f6cec073280 2024-04-24T22:43:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.568+0000 7f6ceb200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6cec15da90 con 0x7f6cec154560 2024-04-24T22:43:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.569+0000 7f6ce9400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6cec073280 legacy=0x7f6cec0cc660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-04-24T22:43:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.569+0000 7f6ce9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 0x7f6cec0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.569+0000 7f6ce9e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 0x7f6cec0cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:45942/0 (socket says 172.21.15.47:45942) 2024-04-24T22:43:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.569+0000 7f6ce9e00640 1 -- 172.21.15.47:0/680638753 learned_addr learned my addr 172.21.15.47:0/680638753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.569+0000 7f6ce9e00640 1 -- 172.21.15.47:0/680638753 >> v1:172.21.15.113:6789/0 conn(0x7f6cec073280 legacy=0x7f6cec0cc660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.569+0000 7f6ce9e00640 1 -- 172.21.15.47:0/680638753 >> v1:172.21.15.47:6789/0 conn(0x7f6cec0cb970 legacy=0x7f6cec0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.569+0000 7f6ce9e00640 1 -- 172.21.15.47:0/680638753 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec15dd70 con 0x7f6cec154560 2024-04-24T22:43:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.571+0000 7f6ce9e00640 1 --2- 172.21.15.47:0/680638753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 0x7f6cec0cb430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0098a0 tx=0x7f6cdc02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e896aed2d708010f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.571+0000 7f6ce8a00640 1 -- 172.21.15.47:0/680638753 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc037070 con 0x7f6cec154560 2024-04-24T22:43:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.571+0000 7f6ce8a00640 1 -- 172.21.15.47:0/680638753 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cdc032440 con 0x7f6cec154560 2024-04-24T22:43:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.571+0000 7f6ce8a00640 1 -- 172.21.15.47:0/680638753 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc032810 con 0x7f6cec154560 2024-04-24T22:43:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.572+0000 7f6ceb200640 1 -- 172.21.15.47:0/680638753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 msgr2=0x7f6cec0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.572+0000 7f6ceb200640 1 --2- 172.21.15.47:0/680638753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 0x7f6cec0cb430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0098a0 tx=0x7f6cdc02a290 comp rx=0 tx=0).stop 2024-04-24T22:43:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.572+0000 7f6ceb200640 1 -- 172.21.15.47:0/680638753 shutdown_connections 2024-04-24T22:43:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.572+0000 7f6ceb200640 1 --2- 172.21.15.47:0/680638753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 0x7f6cec0cb430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.572+0000 7f6ceb200640 1 -- 172.21.15.47:0/680638753 >> 172.21.15.47:0/680638753 conn(0x7f6cec0672c0 msgr2=0x7f6cec072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.572+0000 7f6ceb200640 1 -- 172.21.15.47:0/680638753 shutdown_connections 2024-04-24T22:43:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.572+0000 7f6ceb200640 1 -- 172.21.15.47:0/680638753 wait complete. 2024-04-24T22:43:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.575+0000 7f6ceb200640 1 Processor -- start 2024-04-24T22:43:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.575+0000 7f6ceb200640 1 -- start start 2024-04-24T22:43:24.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.576+0000 7f6ceb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cec073280 0x7f6cec0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.576+0000 7f6ceb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6cec0cb970 0x7f6cec159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.576+0000 7f6ceb200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 0x7f6cec0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.576+0000 7f6ceb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec0caad0 con 0x7f6cec073280 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.576+0000 7f6ceb200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec15e330 con 0x7f6cec0cb970 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.576+0000 7f6ceb200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6cec15ae80 con 0x7f6cec154560 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6ce9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cec073280 0x7f6cec0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6ce9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cec073280 0x7f6cec0ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34922/0 (socket says 172.21.15.47:34922) 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6ce9e00640 1 -- 172.21.15.47:0/652859529 learned_addr learned my addr 172.21.15.47:0/652859529 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6ce9400640 1 -- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6cec0cb970 msgr2=0x7f6cec159510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:3300/0 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6cea800640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 0x7f6cec0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6ce9400640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6cec0cb970 0x7f6cec159510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6cea800640 1 -- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6cec0cb970 msgr2=0x7f6cec159510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6cea800640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6cec0cb970 0x7f6cec159510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6cea800640 1 -- 172.21.15.47:0/652859529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cec073280 msgr2=0x7f6cec0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6cea800640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cec073280 0x7f6cec0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6cea800640 1 -- 172.21.15.47:0/652859529 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec0d0240 con 0x7f6cec154560 2024-04-24T22:43:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6ce9e00640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cec073280 0x7f6cec0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.577+0000 7f6cea800640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 0x7f6cec0cf290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c730 tx=0x7f6ce400cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.578+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4016020 con 0x7f6cec154560 2024-04-24T22:43:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.578+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce4006d50 con 0x7f6cec154560 2024-04-24T22:43:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.578+0000 7f6ceb200640 1 -- 172.21.15.47:0/652859529 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cec0ced90 con 0x7f6cec154560 2024-04-24T22:43:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.578+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4003bb0 con 0x7f6cec154560 2024-04-24T22:43:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.578+0000 7f6ceb200640 1 -- 172.21.15.47:0/652859529 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cec157a40 con 0x7f6cec154560 2024-04-24T22:43:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.579+0000 7f6ceb200640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec156e60 0x7f6cec1f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.579+0000 7f6ceb200640 1 -- 172.21.15.47:0/652859529 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6cec0cec10 con 0x7f6cec156e60 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.580+0000 7f6ce9e00640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec156e60 0x7f6cec1f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.580+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ce400a5e0 con 0x7f6cec154560 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.580+0000 7f6ce9e00640 1 -- 172.21.15.47:0/652859529 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6cec073b90 con 0x7f6cec156e60 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.581+0000 7f6ce9e00640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec156e60 0x7f6cec1f0f50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0097f0 tx=0x7f6cdc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.581+0000 7f6ce2000640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6ccc06bde0 0x7f6ccc06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.581+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6ce408aab0 con 0x7f6cec154560 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.581+0000 7f6ce9400640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6ccc06bde0 0x7f6ccc06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.581+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc043070 con 0x7f6cec156e60 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.581+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6cdc03c850 con 0x7f6cec156e60 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.581+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec156e60 msgr2=0x7f6cec1f0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.581+0000 7f6ce2000640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec156e60 0x7f6cec1f0f50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0097f0 tx=0x7f6cdc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.581+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec156e60 msgr2=0x7f6cec1f0f50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.581+0000 7f6ce2000640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec156e60 0x7f6cec1f0f50 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0097f0 tx=0x7f6cdc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.582+0000 7f6ce9400640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6ccc06bde0 0x7f6ccc06e2a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8006fd0 tx=0x7f6cd8006d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.590+0000 7f6ceb200640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec0cfd00 0x7f6cec1f1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.590+0000 7f6ceb200640 1 -- 172.21.15.47:0/652859529 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6cec1f19d0 con 0x7f6cec0cfd00 2024-04-24T22:43:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.591+0000 7f6ce9e00640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec0cfd00 0x7f6cec1f1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.592+0000 7f6ce9e00640 1 -- 172.21.15.47:0/652859529 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6cec157c40 con 0x7f6cec0cfd00 2024-04-24T22:43:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.594+0000 7f6ce9e00640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec0cfd00 0x7f6cec1f1490 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc02a2d0 tx=0x7f6cdc009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.594+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc037040 con 0x7f6cec0cfd00 2024-04-24T22:43:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.594+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5053 (secure 0 0 0) 0x7f6cdc003b70 con 0x7f6cec0cfd00 2024-04-24T22:43:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.594+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec0cfd00 msgr2=0x7f6cec1f1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.594+0000 7f6ce2000640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec0cfd00 0x7f6cec1f1490 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc02a2d0 tx=0x7f6cdc009240 comp rx=0 tx=0).stop 2024-04-24T22:43:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.594+0000 7f6ce2000640 1 -- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec0cfd00 msgr2=0x7f6cec1f1490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.594+0000 7f6ce2000640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec0cfd00 0x7f6cec1f1490 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc02a2d0 tx=0x7f6cdc009240 comp rx=0 tx=0).stop 2024-04-24T22:43:24.600 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:43:24.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:24.600 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:24.600 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:43:24.600 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 34, 2024-04-24T22:43:24.600 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 38236, 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:24.601 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:24.602 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:43:24.603 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:43:24.604 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:43:24.604 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:43:24.604 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:24.604 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:24.604 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:43:24.604 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:43:24.604 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:43:24.604 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:43:24.604 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:43:24.604 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:24.605 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:43:24.607 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:24.609 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.612 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:43:24.613 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.614 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.615 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:43:24.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.615 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:24.615 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.615 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:43:24.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:24.617 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:24.617 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:24.617 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:24.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:24.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:24.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:24.617 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:43:24.617 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:43:24.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.598+0000 7f6ceb200640 1 -- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6ccc06bde0 msgr2=0x7f6ccc06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.598+0000 7f6ceb200640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f6ccc06bde0 0x7f6ccc06e2a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8006fd0 tx=0x7f6cd8006d80 comp rx=0 tx=0).stop 2024-04-24T22:43:24.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.598+0000 7f6ceb200640 1 -- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 msgr2=0x7f6cec0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:24.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.598+0000 7f6ceb200640 1 --2- 172.21.15.47:0/652859529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6cec154560 0x7f6cec0cf290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c730 tx=0x7f6ce400cbf0 comp rx=0 tx=0).stop 2024-04-24T22:43:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.598+0000 7f6cea800640 1 -- 172.21.15.47:0/652859529 reap_dead start 2024-04-24T22:43:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.598+0000 7f6cea800640 1 -- 172.21.15.47:0/652859529 reap_dead start 2024-04-24T22:43:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.598+0000 7f6ceb200640 1 -- 172.21.15.47:0/652859529 shutdown_connections 2024-04-24T22:43:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.598+0000 7f6ceb200640 1 -- 172.21.15.47:0/652859529 >> 172.21.15.47:0/652859529 conn(0x7f6cec0672c0 msgr2=0x7f6cec1511f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.599+0000 7f6ceb200640 1 -- 172.21.15.47:0/652859529 shutdown_connections 2024-04-24T22:43:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:24.599+0000 7f6ceb200640 1 -- 172.21.15.47:0/652859529 wait complete. 2024-04-24T22:43:24.620 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-24T22:43:24.620 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-24T22:43:31.989 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:43:31.989 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 rewriters = 353518.56 kB/sec 2024-04-24T22:43:31.989 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 rewriters = 353508.12 kB/sec 2024-04-24T22:43:31.989 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 353518.56 kB/sec 2024-04-24T22:43:31.989 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 353518.56 kB/sec 2024-04-24T22:43:31.989 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 353518.56 kB/sec 2024-04-24T22:43:31.990 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 10485760.00 kB 2024-04-24T22:43:35.944 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:43:35.944 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 readers = 5939917.00 kB/sec 2024-04-24T22:43:35.944 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 readers = 5936964.93 kB/sec 2024-04-24T22:43:35.945 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 5939917.00 kB/sec 2024-04-24T22:43:35.945 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 5939917.00 kB/sec 2024-04-24T22:43:35.945 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 5939917.00 kB/sec 2024-04-24T22:43:35.945 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 10485760.00 kB 2024-04-24T22:43:40.774 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:43:40.774 INFO:tasks.workunit.client.0.smithi047.stdout: Children see throughput for 1 re-readers = 3808403.00 kB/sec 2024-04-24T22:43:40.774 INFO:tasks.workunit.client.0.smithi047.stdout: Parent sees throughput for 1 re-readers = 3807140.33 kB/sec 2024-04-24T22:43:40.775 INFO:tasks.workunit.client.0.smithi047.stdout: Min throughput per process = 3808403.00 kB/sec 2024-04-24T22:43:40.775 INFO:tasks.workunit.client.0.smithi047.stdout: Max throughput per process = 3808403.00 kB/sec 2024-04-24T22:43:40.775 INFO:tasks.workunit.client.0.smithi047.stdout: Avg throughput per process = 3808403.00 kB/sec 2024-04-24T22:43:40.775 INFO:tasks.workunit.client.0.smithi047.stdout: Min xfer = 10485760.00 kB 2024-04-24T22:43:41.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:43:41.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:43:44.621 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-24T22:43:44.621 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-04-24T22:43:44.621 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-24T22:43:44.621 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-24T22:43:44.625 INFO:tasks.ceph.mon.b:Started 2024-04-24T22:43:44.625 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-24T22:43:44.625 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:43:44.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.845+0000 7fcafe200640 1 Processor -- start 2024-04-24T22:43:44.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.845+0000 7fcafe200640 1 -- start start 2024-04-24T22:43:44.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.846+0000 7fcafe200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 0x7fcaf806e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:44.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.846+0000 7fcafe200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcaf80c6a20 0x7fcaf80707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:44.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.846+0000 7fcafe200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf8071570 con 0x7fcaf806e6f0 2024-04-24T22:43:44.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.846+0000 7fcafe200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf815b470 con 0x7fcaf80c6a20 2024-04-24T22:43:44.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.846+0000 7fcafe200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf815a8a0 con 0x7fcaf81514f0 2024-04-24T22:43:44.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.846+0000 7fcafce00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 0x7fcaf806e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:44.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.846+0000 7fcafd800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcaf80c6a20 0x7fcaf80707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:44.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.846+0000 7fcaf7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcaf806e6f0 0x7fcaf80c6310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:51792/0 (socket says 172.21.15.47:51792) 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.847+0000 7fcaf7e00640 1 -- 172.21.15.47:0/1508593107 learned_addr learned my addr 172.21.15.47:0/1508593107 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.847+0000 7fcafd800640 1 -- 172.21.15.47:0/1508593107 >> v2:172.21.15.113:3300/0 conn(0x7fcaf80c6a20 msgr2=0x7fcaf80707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.847+0000 7fcafd800640 1 -- 172.21.15.47:0/1508593107 >> v2:172.21.15.113:3300/0 conn(0x7fcaf80c6a20 msgr2=0x7fcaf80707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.847+0000 7fcafd800640 1 --2- 172.21.15.47:0/1508593107 >> v2:172.21.15.113:3300/0 conn(0x7fcaf80c6a20 0x7fcaf80707e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.847+0000 7fcafd800640 1 --2- 172.21.15.47:0/1508593107 >> v2:172.21.15.113:3300/0 conn(0x7fcaf80c6a20 0x7fcaf80707e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.847+0000 7fcafce00640 1 -- 172.21.15.47:0/1508593107 >> v2:172.21.15.113:3300/0 conn(0x7fcaf80c6a20 msgr2=0x7fcaf80707e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.847+0000 7fcafce00640 1 --2- 172.21.15.47:0/1508593107 >> v2:172.21.15.113:3300/0 conn(0x7fcaf80c6a20 0x7fcaf80707e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.847+0000 7fcafce00640 1 -- 172.21.15.47:0/1508593107 >> v1:172.21.15.113:6789/0 conn(0x7fcaf806e6f0 legacy=0x7fcaf80c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.847+0000 7fcafce00640 1 -- 172.21.15.47:0/1508593107 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf81518d0 con 0x7fcaf81514f0 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.848+0000 7fcafce00640 1 --2- 172.21.15.47:0/1508593107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 0x7fcaf806e1b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009d90 tx=0x7fcaec02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b5b02091a5ae7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.849+0000 7fcaf7400640 1 -- 172.21.15.47:0/1508593107 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec007740 con 0x7fcaf81514f0 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.849+0000 7fcaf7400640 1 -- 172.21.15.47:0/1508593107 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec0078e0 con 0x7fcaf81514f0 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.849+0000 7fcaf7400640 1 -- 172.21.15.47:0/1508593107 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec007cb0 con 0x7fcaf81514f0 2024-04-24T22:43:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.849+0000 7fcafe200640 1 -- 172.21.15.47:0/1508593107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 msgr2=0x7fcaf806e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.849+0000 7fcafe200640 1 --2- 172.21.15.47:0/1508593107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 0x7fcaf806e1b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009d90 tx=0x7fcaec02a660 comp rx=0 tx=0).stop 2024-04-24T22:43:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.849+0000 7fcafe200640 1 -- 172.21.15.47:0/1508593107 shutdown_connections 2024-04-24T22:43:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.849+0000 7fcafe200640 1 --2- 172.21.15.47:0/1508593107 >> v2:172.21.15.113:3300/0 conn(0x7fcaf80c6a20 0x7fcaf80707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.849+0000 7fcafe200640 1 --2- 172.21.15.47:0/1508593107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 0x7fcaf806e1b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.849+0000 7fcafe200640 1 -- 172.21.15.47:0/1508593107 >> 172.21.15.47:0/1508593107 conn(0x7fcaf80cb900 msgr2=0x7fcaf80cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.849+0000 7fcafe200640 1 -- 172.21.15.47:0/1508593107 shutdown_connections 2024-04-24T22:43:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.850+0000 7fcafe200640 1 -- 172.21.15.47:0/1508593107 wait complete. 2024-04-24T22:43:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.851+0000 7fcafe200640 1 Processor -- start 2024-04-24T22:43:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.852+0000 7fcafe200640 1 -- start start 2024-04-24T22:43:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcafe200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf806e6f0 0x7fcaf81f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcafe200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf80c6a20 0x7fcaf814f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcafe200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 0x7fcaf8159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcafe200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf815b100 con 0x7fcaf806e6f0 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcafe200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf815ae20 con 0x7fcaf80c6a20 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcafe200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf80c7ba0 con 0x7fcaf81514f0 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcafce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf806e6f0 0x7fcaf81f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcafce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf806e6f0 0x7fcaf81f0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35208/0 (socket says 172.21.15.47:35208) 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcafce00640 1 -- 172.21.15.47:0/257457556 learned_addr learned my addr 172.21.15.47:0/257457556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcafd800640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 0x7fcaf8159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.853+0000 7fcaf7e00640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf80c6a20 0x7fcaf814f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcaf7e00640 1 -- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf80c6a20 msgr2=0x7fcaf814f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcaf7e00640 1 -- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf80c6a20 msgr2=0x7fcaf814f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcaf7e00640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf80c6a20 0x7fcaf814f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcaf7e00640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf80c6a20 0x7fcaf814f3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcafd800640 1 -- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf80c6a20 msgr2=0x7fcaf814f3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:43:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcafd800640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf80c6a20 0x7fcaf814f3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcafd800640 1 -- 172.21.15.47:0/257457556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf806e6f0 msgr2=0x7fcaf81f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcafd800640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf806e6f0 0x7fcaf81f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcafd800640 1 -- 172.21.15.47:0/257457556 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf81f3950 con 0x7fcaf81514f0 2024-04-24T22:43:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcafd800640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 0x7fcaf8159370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000d760 tx=0x7fcaf000dc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.854+0000 7fcaf4c00640 1 -- 172.21.15.47:0/257457556 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0015070 con 0x7fcaf81514f0 2024-04-24T22:43:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.855+0000 7fcaf4c00640 1 -- 172.21.15.47:0/257457556 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf0004220 con 0x7fcaf81514f0 2024-04-24T22:43:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.855+0000 7fcafe200640 1 -- 172.21.15.47:0/257457556 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf81e09c0 con 0x7fcaf81514f0 2024-04-24T22:43:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.855+0000 7fcaf4c00640 1 -- 172.21.15.47:0/257457556 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000de70 con 0x7fcaf81514f0 2024-04-24T22:43:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.855+0000 7fcafe200640 1 -- 172.21.15.47:0/257457556 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf81f3ed0 con 0x7fcaf81514f0 2024-04-24T22:43:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.856+0000 7fcafe200640 1 -- 172.21.15.47:0/257457556 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf8068d50 con 0x7fcaf81514f0 2024-04-24T22:43:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.857+0000 7fcaf4c00640 1 -- 172.21.15.47:0/257457556 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcaf00078b0 con 0x7fcaf81514f0 2024-04-24T22:43:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.858+0000 7fcaf4c00640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcad806be30 0x7fcad806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.858+0000 7fcaf4c00640 1 -- 172.21.15.47:0/257457556 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcaf008af60 con 0x7fcaf81514f0 2024-04-24T22:43:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.858+0000 7fcafce00640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcad806be30 0x7fcad806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.859+0000 7fcafce00640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcad806be30 0x7fcad806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009bd0 tx=0x7fcaec02f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:44.862+0000 7fcaf4c00640 1 -- 172.21.15.47:0/257457556 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcaf0059110 con 0x7fcaf81514f0 2024-04-24T22:43:45.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.140+0000 7fcafe200640 1 -- 172.21.15.47:0/257457556 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcaf80720d0 con 0x7fcaf81514f0 2024-04-24T22:43:45.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.141+0000 7fcaf4c00640 1 -- 172.21.15.47:0/257457556 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1559 (secure 0 0 0) 0x7fcaf005d410 con 0x7fcaf81514f0 2024-04-24T22:43:45.142 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":34,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:43:45.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.143+0000 7fcafe200640 1 -- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcad806be30 msgr2=0x7fcad806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.143+0000 7fcafe200640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcad806be30 0x7fcad806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009bd0 tx=0x7fcaec02f000 comp rx=0 tx=0).stop 2024-04-24T22:43:45.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.143+0000 7fcafe200640 1 -- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 msgr2=0x7fcaf8159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:45.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.143+0000 7fcafe200640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 0x7fcaf8159370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000d760 tx=0x7fcaf000dc20 comp rx=0 tx=0).stop 2024-04-24T22:43:45.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.143+0000 7fcafe200640 1 -- 172.21.15.47:0/257457556 shutdown_connections 2024-04-24T22:43:45.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.143+0000 7fcafe200640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fcad806be30 0x7fcad806e2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.143+0000 7fcafe200640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcaf81514f0 0x7fcaf8159370 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.143+0000 7fcafe200640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcaf80c6a20 0x7fcaf814f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.143+0000 7fcafe200640 1 --2- 172.21.15.47:0/257457556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcaf806e6f0 0x7fcaf81f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:45.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.143+0000 7fcafe200640 1 -- 172.21.15.47:0/257457556 >> 172.21.15.47:0/257457556 conn(0x7fcaf80cb900 msgr2=0x7fcaf8072d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:45.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.144+0000 7fcafe200640 1 -- 172.21.15.47:0/257457556 shutdown_connections 2024-04-24T22:43:45.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:45.144+0000 7fcafe200640 1 -- 172.21.15.47:0/257457556 wait complete. 2024-04-24T22:43:48.165 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:43:48.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.413+0000 7f79f2400640 1 Processor -- start 2024-04-24T22:43:48.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.413+0000 7f79f2400640 1 -- start start 2024-04-24T22:43:48.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.415+0000 7f79f2400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 0x7f79ec0c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:48.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.415+0000 7f79f2400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f79ec072b80 0x7f79ec073010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.415+0000 7f79f2400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79ec0c7bb0 con 0x7f79ec152960 2024-04-24T22:43:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.415+0000 7f79f2400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f79ec158f20 con 0x7f79ec072b80 2024-04-24T22:43:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.415+0000 7f79f2400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f79ec1593b0 con 0x7f79ec0c7750 2024-04-24T22:43:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.415+0000 7f79eb400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f79ec152960 0x7f79ec15ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:42878/0 (socket says 172.21.15.47:42878) 2024-04-24T22:43:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.415+0000 7f79eb400640 1 -- 172.21.15.47:0/1273968827 learned_addr learned my addr 172.21.15.47:0/1273968827 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:43:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.415+0000 7f79ebe00640 1 --2- 172.21.15.47:0/1273968827 >> v2:172.21.15.47:3300/0 conn(0x7f79ec072b80 0x7f79ec073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.415+0000 7f79eaa00640 1 --2- 172.21.15.47:0/1273968827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 0x7f79ec0c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.416+0000 7f79ebe00640 1 -- 172.21.15.47:0/1273968827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 msgr2=0x7f79ec0c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.416+0000 7f79ebe00640 1 --2- 172.21.15.47:0/1273968827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 0x7f79ec0c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.416+0000 7f79ebe00640 1 -- 172.21.15.47:0/1273968827 >> v1:172.21.15.113:6789/0 conn(0x7f79ec152960 legacy=0x7f79ec15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.416+0000 7f79ebe00640 1 -- 172.21.15.47:0/1273968827 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec0c7e50 con 0x7f79ec072b80 2024-04-24T22:43:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.418+0000 7f79ebe00640 1 --2- 172.21.15.47:0/1273968827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ec072b80 0x7f79ec073010 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009d90 tx=0x7f79e402eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a1e5f21b7aa616 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.418+0000 7f79ea000640 1 -- 172.21.15.47:0/1273968827 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4030860 con 0x7f79ec072b80 2024-04-24T22:43:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.418+0000 7f79ea000640 1 -- 172.21.15.47:0/1273968827 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e4030a00 con 0x7f79ec072b80 2024-04-24T22:43:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.418+0000 7f79ea000640 1 -- 172.21.15.47:0/1273968827 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4030dd0 con 0x7f79ec072b80 2024-04-24T22:43:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.419+0000 7f79f2400640 1 -- 172.21.15.47:0/1273968827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ec072b80 msgr2=0x7f79ec073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.419+0000 7f79f2400640 1 --2- 172.21.15.47:0/1273968827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ec072b80 0x7f79ec073010 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009d90 tx=0x7f79e402eb00 comp rx=0 tx=0).stop 2024-04-24T22:43:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.419+0000 7f79f2400640 1 -- 172.21.15.47:0/1273968827 shutdown_connections 2024-04-24T22:43:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.419+0000 7f79f2400640 1 --2- 172.21.15.47:0/1273968827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ec072b80 0x7f79ec073010 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.419+0000 7f79f2400640 1 --2- 172.21.15.47:0/1273968827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 0x7f79ec0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.419+0000 7f79f2400640 1 -- 172.21.15.47:0/1273968827 >> 172.21.15.47:0/1273968827 conn(0x7f79ec06e580 msgr2=0x7f79ec06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.419+0000 7f79f2400640 1 -- 172.21.15.47:0/1273968827 shutdown_connections 2024-04-24T22:43:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.419+0000 7f79f2400640 1 -- 172.21.15.47:0/1273968827 wait complete. 2024-04-24T22:43:48.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.421+0000 7f79f2400640 1 Processor -- start 2024-04-24T22:43:48.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.422+0000 7f79f2400640 1 -- start start 2024-04-24T22:43:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79f2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec072b80 0x7f79ec15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79f2400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 0x7f79ec070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79f2400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ec152960 0x7f79ec071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79f2400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec1588d0 con 0x7f79ec152960 2024-04-24T22:43:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79f2400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec0c8130 con 0x7f79ec072b80 2024-04-24T22:43:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79f2400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f79ec158bb0 con 0x7f79ec0c7750 2024-04-24T22:43:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79eb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec072b80 0x7f79ec15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79ebe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ec152960 0x7f79ec071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79eb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec072b80 0x7f79ec15be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:50324/0 (socket says 172.21.15.47:50324) 2024-04-24T22:43:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79eb400640 1 -- 172.21.15.47:0/1556813078 learned_addr learned my addr 172.21.15.47:0/1556813078 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79eaa00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 0x7f79ec070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79eb400640 1 -- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec072b80 msgr2=0x7f79ec15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-24T22:43:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79eb400640 1 -- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec072b80 msgr2=0x7f79ec15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:43:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79eb400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec072b80 0x7f79ec15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79eb400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec072b80 0x7f79ec15be70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79ebe00640 1 -- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 msgr2=0x7f79ec070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79ebe00640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 0x7f79ec070dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79ebe00640 1 -- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec072b80 msgr2=0x7f79ec15be70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79ebe00640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec072b80 0x7f79ec15be70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.423+0000 7f79ebe00640 1 -- 172.21.15.47:0/1556813078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec15c3b0 con 0x7f79ec152960 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.424+0000 7f79eaa00640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 0x7f79ec070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.424+0000 7f79ebe00640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ec152960 0x7f79ec071310 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f79e4036000 tx=0x7f79e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.424+0000 7f79e3400640 1 -- 172.21.15.47:0/1556813078 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4031b70 con 0x7f79ec152960 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.424+0000 7f79e3400640 1 -- 172.21.15.47:0/1556813078 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e4049070 con 0x7f79ec152960 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.424+0000 7f79e3400640 1 -- 172.21.15.47:0/1556813078 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e403a400 con 0x7f79ec152960 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.425+0000 7f79f2400640 1 -- 172.21.15.47:0/1556813078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ec0c55a0 con 0x7f79ec152960 2024-04-24T22:43:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.425+0000 7f79f2400640 1 -- 172.21.15.47:0/1556813078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ec148a10 con 0x7f79ec152960 2024-04-24T22:43:48.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.427+0000 7f79f2400640 1 -- 172.21.15.47:0/1556813078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ec073b90 con 0x7f79ec152960 2024-04-24T22:43:48.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.428+0000 7f79e3400640 1 -- 172.21.15.47:0/1556813078 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79e403a5a0 con 0x7f79ec152960 2024-04-24T22:43:48.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.429+0000 7f79e3400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79cc06be80 0x7f79cc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:48.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.429+0000 7f79e3400640 1 -- 172.21.15.47:0/1556813078 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f79e40b01f0 con 0x7f79ec152960 2024-04-24T22:43:48.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.430+0000 7f79eb400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79cc06be80 0x7f79cc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:48.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.430+0000 7f79eb400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79cc06be80 0x7f79cc06e340 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79dc005880 tx=0x7f79dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:48.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.436+0000 7f79e3400640 1 -- 172.21.15.47:0/1556813078 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f79e4041070 con 0x7f79ec152960 2024-04-24T22:43:48.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.810+0000 7f79f2400640 1 -- 172.21.15.47:0/1556813078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f79ec0c92a0 con 0x7f79ec152960 2024-04-24T22:43:48.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.811+0000 7f79e3400640 1 -- 172.21.15.47:0/1556813078 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1559 (secure 0 0 0) 0x7f79e4036030 con 0x7f79ec152960 2024-04-24T22:43:48.813 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":34,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:43:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.813+0000 7f79f2400640 1 -- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79cc06be80 msgr2=0x7f79cc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.813+0000 7f79f2400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79cc06be80 0x7f79cc06e340 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79dc005880 tx=0x7f79dc009040 comp rx=0 tx=0).stop 2024-04-24T22:43:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.813+0000 7f79f2400640 1 -- 172.21.15.47:0/1556813078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ec152960 msgr2=0x7f79ec071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.813+0000 7f79f2400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ec152960 0x7f79ec071310 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f79e4036000 tx=0x7f79e4004a40 comp rx=0 tx=0).stop 2024-04-24T22:43:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.814+0000 7f79f2400640 1 -- 172.21.15.47:0/1556813078 shutdown_connections 2024-04-24T22:43:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.814+0000 7f79f2400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f79cc06be80 0x7f79cc06e340 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.814+0000 7f79f2400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ec152960 0x7f79ec071310 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.814+0000 7f79f2400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79ec0c7750 0x7f79ec070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.814+0000 7f79f2400640 1 --2- 172.21.15.47:0/1556813078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec072b80 0x7f79ec15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.814+0000 7f79f2400640 1 -- 172.21.15.47:0/1556813078 >> 172.21.15.47:0/1556813078 conn(0x7f79ec06e580 msgr2=0x7f79ec0c61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.814+0000 7f79f2400640 1 -- 172.21.15.47:0/1556813078 shutdown_connections 2024-04-24T22:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:48.814+0000 7f79f2400640 1 -- 172.21.15.47:0/1556813078 wait complete. 2024-04-24T22:43:49.359 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:43:49.359 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:43:49.359 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-04-24T22:43:49.359 INFO:tasks.workunit.client.0.smithi047.stdout:iozone test complete. 2024-04-24T22:43:49.361 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T22:43:49.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-24T22:43:49.463 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-04-24T22:43:49.463 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T22:43:50.591 DEBUG:teuthology.parallel:result is None 2024-04-24T22:43:50.591 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T22:43:50.681 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T22:43:50.682 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-04-24T22:43:50.692 INFO:tasks.mon_thrash:joining mon_thrasher 2024-04-24T22:43:51.835 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:43:52.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.052+0000 7f00cc800640 1 Processor -- start 2024-04-24T22:43:52.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.052+0000 7f00cc800640 1 -- start start 2024-04-24T22:43:52.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.054+0000 7f00cc800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00c4151780 0x7f00c40c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.054+0000 7f00cc800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4073670 0x7f00c40c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.054+0000 7f00cc800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c415aa30 con 0x7f00c40c1a00 2024-04-24T22:43:52.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.054+0000 7f00cc800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f00c415ad10 con 0x7f00c4151780 2024-04-24T22:43:52.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.054+0000 7f00cc800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f00c415aff0 con 0x7f00c4073670 2024-04-24T22:43:52.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.054+0000 7f00c9400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f00c40c1a00 0x7f00c4072f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37336/0 (socket says 172.21.15.47:37336) 2024-04-24T22:43:52.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.054+0000 7f00c9400640 1 -- 172.21.15.47:0/3456149096 learned_addr learned my addr 172.21.15.47:0/3456149096 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:43:52.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.054+0000 7f00c9e00640 1 --2- 172.21.15.47:0/3456149096 >> v2:172.21.15.113:3300/0 conn(0x7f00c4151780 0x7f00c40c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.054+0000 7f00ca800640 1 --2- 172.21.15.47:0/3456149096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4073670 0x7f00c40c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.055+0000 7f00c9e00640 1 -- 172.21.15.47:0/3456149096 >> v2:172.21.15.113:3300/0 conn(0x7f00c4151780 msgr2=0x7f00c40c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T22:43:52.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.055+0000 7f00c9e00640 1 -- 172.21.15.47:0/3456149096 >> v2:172.21.15.113:3300/0 conn(0x7f00c4151780 msgr2=0x7f00c40c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:52.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.055+0000 7f00c9e00640 1 --2- 172.21.15.47:0/3456149096 >> v2:172.21.15.113:3300/0 conn(0x7f00c4151780 0x7f00c40c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:52.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.055+0000 7f00c9e00640 1 --2- 172.21.15.47:0/3456149096 >> v2:172.21.15.113:3300/0 conn(0x7f00c4151780 0x7f00c40c8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:52.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.055+0000 7f00ca800640 1 -- 172.21.15.47:0/3456149096 >> v2:172.21.15.113:3300/0 conn(0x7f00c4151780 msgr2=0x7f00c40c8590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:52.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.055+0000 7f00ca800640 1 --2- 172.21.15.47:0/3456149096 >> v2:172.21.15.113:3300/0 conn(0x7f00c4151780 0x7f00c40c8590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.055+0000 7f00ca800640 1 -- 172.21.15.47:0/3456149096 >> v1:172.21.15.47:6789/0 conn(0x7f00c40c1a00 legacy=0x7f00c4072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.055+0000 7f00c8a00640 1 -- 172.21.15.47:0/3456149096 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3444073363 0 0) 0x7f00c415aa30 con 0x7f00c40c1a00 2024-04-24T22:43:52.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.055+0000 7f00ca800640 1 -- 172.21.15.47:0/3456149096 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c415b2d0 con 0x7f00c4073670 2024-04-24T22:43:52.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.056+0000 7f00ca800640 1 --2- 172.21.15.47:0/3456149096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4073670 0x7f00c40c8ad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009d90 tx=0x7f00c002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b469a32e350cbd31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.056+0000 7f00c8a00640 1 -- 172.21.15.47:0/3456149096 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0007740 con 0x7f00c4073670 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.056+0000 7f00c8a00640 1 -- 172.21.15.47:0/3456149096 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c00078e0 con 0x7f00c4073670 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.056+0000 7f00c8a00640 1 -- 172.21.15.47:0/3456149096 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0007cb0 con 0x7f00c4073670 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.057+0000 7f00cc800640 1 -- 172.21.15.47:0/3456149096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4073670 msgr2=0x7f00c40c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.057+0000 7f00cc800640 1 --2- 172.21.15.47:0/3456149096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4073670 0x7f00c40c8ad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009d90 tx=0x7f00c002a6f0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.057+0000 7f00cc800640 1 -- 172.21.15.47:0/3456149096 shutdown_connections 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.057+0000 7f00cc800640 1 --2- 172.21.15.47:0/3456149096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4073670 0x7f00c40c8ad0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.057+0000 7f00cc800640 1 --2- 172.21.15.47:0/3456149096 >> v2:172.21.15.113:3300/0 conn(0x7f00c4151780 0x7f00c40c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.057+0000 7f00cc800640 1 -- 172.21.15.47:0/3456149096 >> 172.21.15.47:0/3456149096 conn(0x7f00c406e580 msgr2=0x7f00c406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.057+0000 7f00cc800640 1 -- 172.21.15.47:0/3456149096 shutdown_connections 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.057+0000 7f00cc800640 1 -- 172.21.15.47:0/3456149096 wait complete. 2024-04-24T22:43:52.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.059+0000 7f00cc800640 1 Processor -- start 2024-04-24T22:43:52.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.059+0000 7f00cc800640 1 -- start start 2024-04-24T22:43:52.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.060+0000 7f00cc800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c4073670 0x7f00c40b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.060+0000 7f00cc800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00c40c1a00 0x7f00c41457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.060+0000 7f00cc800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4151780 0x7f00c4145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.060+0000 7f00cc800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f00c4157870 con 0x7f00c40c1a00 2024-04-24T22:43:52.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.060+0000 7f00cc800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41576f0 con 0x7f00c4073670 2024-04-24T22:43:52.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.060+0000 7f00cc800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f00c41579f0 con 0x7f00c4151780 2024-04-24T22:43:52.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00c9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00c40c1a00 0x7f00c41457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00ca800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4151780 0x7f00c4145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00c9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00c40c1a00 0x7f00c41457e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48116/0 (socket says 172.21.15.47:48116) 2024-04-24T22:43:52.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00c9400640 1 -- 172.21.15.47:0/2035633185 learned_addr learned my addr 172.21.15.47:0/2035633185 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:52.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00c9e00640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c4073670 0x7f00c40b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00c9e00640 1 -- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c4073670 msgr2=0x7f00c40b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-24T22:43:52.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00c9e00640 1 -- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c4073670 msgr2=0x7f00c40b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T22:43:52.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00c9e00640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c4073670 0x7f00c40b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:52.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00c9e00640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c4073670 0x7f00c40b0ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:52.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00ca800640 1 -- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c4073670 msgr2=0x7f00c40b0ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T22:43:52.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00ca800640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c4073670 0x7f00c40b0ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00ca800640 1 -- 172.21.15.47:0/2035633185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00c40c1a00 msgr2=0x7f00c41457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00ca800640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00c40c1a00 0x7f00c41457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00ca800640 1 -- 172.21.15.47:0/2035633185 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c40b1220 con 0x7f00c4151780 2024-04-24T22:43:52.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.061+0000 7f00c9400640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00c40c1a00 0x7f00c41457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:52.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.062+0000 7f00ca800640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4151780 0x7f00c4145d20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f00c0031000 tx=0x7f00c0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.062+0000 7f00be000640 1 -- 172.21.15.47:0/2035633185 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0041070 con 0x7f00c4151780 2024-04-24T22:43:52.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.062+0000 7f00be000640 1 -- 172.21.15.47:0/2035633185 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c0031bc0 con 0x7f00c4151780 2024-04-24T22:43:52.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.062+0000 7f00be000640 1 -- 172.21.15.47:0/2035633185 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0009450 con 0x7f00c4151780 2024-04-24T22:43:52.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.062+0000 7f00cc800640 1 -- 172.21.15.47:0/2035633185 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c41f0500 con 0x7f00c4151780 2024-04-24T22:43:52.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.062+0000 7f00cc800640 1 -- 172.21.15.47:0/2035633185 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c41f0ac0 con 0x7f00c4151780 2024-04-24T22:43:52.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.063+0000 7f00cc800640 1 -- 172.21.15.47:0/2035633185 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c4073b90 con 0x7f00c4151780 2024-04-24T22:43:52.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.064+0000 7f00be000640 1 -- 172.21.15.47:0/2035633185 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00c0039040 con 0x7f00c4151780 2024-04-24T22:43:52.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.065+0000 7f00be000640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f00a806be30 0x7f00a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.065+0000 7f00be000640 1 -- 172.21.15.47:0/2035633185 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f00c00ac3c0 con 0x7f00c4151780 2024-04-24T22:43:52.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.066+0000 7f00c9e00640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f00a806be30 0x7f00a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.066+0000 7f00c9e00640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f00a806be30 0x7f00a806e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f00b8006fd0 tx=0x7f00b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.070+0000 7f00be000640 1 -- 172.21.15.47:0/2035633185 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00c007a570 con 0x7f00c4151780 2024-04-24T22:43:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.353+0000 7f00cc800640 1 -- 172.21.15.47:0/2035633185 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f00c40c6740 con 0x7f00c4151780 2024-04-24T22:43:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.354+0000 7f00be000640 1 -- 172.21.15.47:0/2035633185 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f00c007e870 con 0x7f00c4151780 2024-04-24T22:43:52.357 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 -- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f00a806be30 msgr2=0x7f00a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f00a806be30 0x7f00a806e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f00b8006fd0 tx=0x7f00b8008040 comp rx=0 tx=0).stop 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 -- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4151780 msgr2=0x7f00c4145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4151780 0x7f00c4145d20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f00c0031000 tx=0x7f00c0004a40 comp rx=0 tx=0).stop 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 -- 172.21.15.47:0/2035633185 shutdown_connections 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f00a806be30 0x7f00a806e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00c4151780 0x7f00c4145d20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00c40c1a00 0x7f00c41457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 --2- 172.21.15.47:0/2035633185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00c4073670 0x7f00c40b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 -- 172.21.15.47:0/2035633185 >> 172.21.15.47:0/2035633185 conn(0x7f00c406e580 msgr2=0x7f00c406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 -- 172.21.15.47:0/2035633185 shutdown_connections 2024-04-24T22:43:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.358+0000 7f00cc800640 1 -- 172.21.15.47:0/2035633185 wait complete. 2024-04-24T22:43:52.381 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-24T22:43:52.381 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-24T22:43:52.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-24T22:43:52.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.591+0000 7f92e3000640 1 Processor -- start 2024-04-24T22:43:52.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.591+0000 7f92e3000640 1 -- start start 2024-04-24T22:43:52.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.592+0000 7f92e3000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92dc0c6bc0 0x7f92dc15a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.592+0000 7f92e3000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92dc0c7d20 0x7f92dc066ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.592+0000 7f92e3000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92dc0cb690 con 0x7f92dc0c67e0 2024-04-24T22:43:52.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.592+0000 7f92e3000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f92dc15aeb0 con 0x7f92dc0c7d20 2024-04-24T22:43:52.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.592+0000 7f92e3000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f92dc15b190 con 0x7f92dc0c6bc0 2024-04-24T22:43:52.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e2600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92dc0c7d20 0x7f92dc066ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e2600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92dc0c7d20 0x7f92dc066ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48122/0 (socket says 172.21.15.47:48122) 2024-04-24T22:43:52.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e2600640 1 -- 172.21.15.47:0/107533892 learned_addr learned my addr 172.21.15.47:0/107533892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e1200640 1 --2- 172.21.15.47:0/107533892 >> v2:172.21.15.113:3300/0 conn(0x7f92dc0c6bc0 0x7f92dc15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e1200640 1 -- 172.21.15.47:0/107533892 >> v2:172.21.15.113:3300/0 conn(0x7f92dc0c6bc0 msgr2=0x7f92dc15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-24T22:43:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e1200640 1 -- 172.21.15.47:0/107533892 >> v2:172.21.15.113:3300/0 conn(0x7f92dc0c6bc0 msgr2=0x7f92dc15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T22:43:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e1200640 1 --2- 172.21.15.47:0/107533892 >> v2:172.21.15.113:3300/0 conn(0x7f92dc0c6bc0 0x7f92dc15a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T22:43:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e1200640 1 --2- 172.21.15.47:0/107533892 >> v2:172.21.15.113:3300/0 conn(0x7f92dc0c6bc0 0x7f92dc15a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T22:43:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e2600640 1 -- 172.21.15.47:0/107533892 >> v2:172.21.15.113:3300/0 conn(0x7f92dc0c6bc0 msgr2=0x7f92dc15a590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T22:43:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e2600640 1 --2- 172.21.15.47:0/107533892 >> v2:172.21.15.113:3300/0 conn(0x7f92dc0c6bc0 0x7f92dc15a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e2600640 1 -- 172.21.15.47:0/107533892 >> v1:172.21.15.113:6789/0 conn(0x7f92dc0c67e0 legacy=0x7f92dc0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.593+0000 7f92e2600640 1 -- 172.21.15.47:0/107533892 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc0cb930 con 0x7f92dc0c7d20 2024-04-24T22:43:52.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.594+0000 7f92e2600640 1 --2- 172.21.15.47:0/107533892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0c7d20 0x7f92dc066ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f92dc071040 tx=0x7f92d402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff0924f966c0d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.595+0000 7f92dbe00640 1 -- 172.21.15.47:0/107533892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4037070 con 0x7f92dc0c7d20 2024-04-24T22:43:52.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.595+0000 7f92dbe00640 1 -- 172.21.15.47:0/107533892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d402ae50 con 0x7f92dc0c7d20 2024-04-24T22:43:52.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.595+0000 7f92dbe00640 1 -- 172.21.15.47:0/107533892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4032860 con 0x7f92dc0c7d20 2024-04-24T22:43:52.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.596+0000 7f92e3000640 1 -- 172.21.15.47:0/107533892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0c7d20 msgr2=0x7f92dc066ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.596+0000 7f92e3000640 1 --2- 172.21.15.47:0/107533892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0c7d20 0x7f92dc066ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f92dc071040 tx=0x7f92d402a300 comp rx=0 tx=0).stop 2024-04-24T22:43:52.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.596+0000 7f92e3000640 1 -- 172.21.15.47:0/107533892 shutdown_connections 2024-04-24T22:43:52.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.596+0000 7f92e3000640 1 --2- 172.21.15.47:0/107533892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0c7d20 0x7f92dc066ea0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.596+0000 7f92e3000640 1 --2- 172.21.15.47:0/107533892 >> v2:172.21.15.113:3300/0 conn(0x7f92dc0c6bc0 0x7f92dc15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.596+0000 7f92e3000640 1 -- 172.21.15.47:0/107533892 >> 172.21.15.47:0/107533892 conn(0x7f92dc0c9410 msgr2=0x7f92dc0c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:52.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.596+0000 7f92e3000640 1 -- 172.21.15.47:0/107533892 shutdown_connections 2024-04-24T22:43:52.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.596+0000 7f92e3000640 1 -- 172.21.15.47:0/107533892 wait complete. 2024-04-24T22:43:52.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.598+0000 7f92e3000640 1 Processor -- start 2024-04-24T22:43:52.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.598+0000 7f92e3000640 1 -- start start 2024-04-24T22:43:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e3000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92dc0c67e0 0x7f92dc156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e3000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92dc0c6bc0 0x7f92dc0bdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e3000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0c7d20 0x7f92dc0be430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e3000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc15aad0 con 0x7f92dc0c7d20 2024-04-24T22:43:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e3000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc15b470 con 0x7f92dc0c67e0 2024-04-24T22:43:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e3000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f92dc15ac50 con 0x7f92dc0c6bc0 2024-04-24T22:43:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92dc0c67e0 0x7f92dc156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e2600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0c7d20 0x7f92dc0be430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e1200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92dc0c6bc0 0x7f92dc0bdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92dc0c67e0 0x7f92dc156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:50366/0 (socket says 172.21.15.47:50366) 2024-04-24T22:43:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.599+0000 7f92e1c00640 1 -- 172.21.15.47:0/2815561543 learned_addr learned my addr 172.21.15.47:0/2815561543 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92e1c00640 1 -- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92dc0c6bc0 msgr2=0x7f92dc0bdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92e1c00640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92dc0c6bc0 0x7f92dc0bdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92e1c00640 1 -- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0c7d20 msgr2=0x7f92dc0be430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92e1c00640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0c7d20 0x7f92dc0be430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92e1c00640 1 -- 172.21.15.47:0/2815561543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc155ce0 con 0x7f92dc0c67e0 2024-04-24T22:43:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92e1200640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92dc0c6bc0 0x7f92dc0bdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:43:52.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92e1c00640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92dc0c67e0 0x7f92dc156fa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00c430 tx=0x7f92cc00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0052d0 con 0x7f92dc0c67e0 2024-04-24T22:43:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92cc005490 con 0x7f92dc0c67e0 2024-04-24T22:43:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc007260 con 0x7f92dc0c67e0 2024-04-24T22:43:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.600+0000 7f92e3000640 1 -- 172.21.15.47:0/2815561543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc155fc0 con 0x7f92dc0c67e0 2024-04-24T22:43:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.601+0000 7f92e3000640 1 -- 172.21.15.47:0/2815561543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc156580 con 0x7f92dc0c67e0 2024-04-24T22:43:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.601+0000 7f92e3000640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0bd2e0 0x7f92dc150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.601+0000 7f92e3000640 1 -- 172.21.15.47:0/2815561543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc0c08e0 con 0x7f92dc0bd2e0 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.602+0000 7f92e1200640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0bd2e0 0x7f92dc150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.602+0000 7f92e1200640 1 -- 172.21.15.47:0/2815561543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92dc15a420 con 0x7f92dc0bd2e0 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.602+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92cc007400 con 0x7f92dc0c67e0 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.602+0000 7f92e1200640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0bd2e0 0x7f92dc150260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f92dc156cc0 tx=0x7f92c800f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.603+0000 7f92d9600640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f92bc06be30 0x7f92bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.604+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f92cc08a480 con 0x7f92dc0c67e0 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.604+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c80147e0 con 0x7f92dc0bd2e0 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.604+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f92c801a070 con 0x7f92dc0bd2e0 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.604+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0bd2e0 msgr2=0x7f92dc150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.604+0000 7f92d9600640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0bd2e0 0x7f92dc150260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f92dc156cc0 tx=0x7f92c800f950 comp rx=0 tx=0).stop 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.604+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0bd2e0 msgr2=0x7f92dc150260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.604+0000 7f92d9600640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc0bd2e0 0x7f92dc150260 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f92dc156cc0 tx=0x7f92c800f950 comp rx=0 tx=0).stop 2024-04-24T22:43:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.604+0000 7f92e2600640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f92bc06be30 0x7f92bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.605+0000 7f92e2600640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f92bc06be30 0x7f92bc06e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f92d4034000 tx=0x7f92d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.611+0000 7f92e3000640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc157e50 0x7f92dc1507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.611+0000 7f92e3000640 1 -- 172.21.15.47:0/2815561543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc0bdd40 con 0x7f92dc157e50 2024-04-24T22:43:52.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.611+0000 7f92e1200640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc157e50 0x7f92dc1507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.612+0000 7f92e1200640 1 -- 172.21.15.47:0/2815561543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f92dc04f5e0 con 0x7f92dc157e50 2024-04-24T22:43:52.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.612+0000 7f92e1200640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc157e50 0x7f92dc1507a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92dc157990 tx=0x7f92c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.612+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8015880 con 0x7f92dc157e50 2024-04-24T22:43:52.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5210 (secure 0 0 0) 0x7f92c8015880 con 0x7f92dc157e50 2024-04-24T22:43:52.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc157e50 msgr2=0x7f92dc1507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92d9600640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc157e50 0x7f92dc1507a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92dc157990 tx=0x7f92c8004a40 comp rx=0 tx=0).stop 2024-04-24T22:43:52.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92d9600640 1 -- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc157e50 msgr2=0x7f92dc1507a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:52.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92d9600640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92dc157e50 0x7f92dc1507a0 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92dc157990 tx=0x7f92c8004a40 comp rx=0 tx=0).stop 2024-04-24T22:43:52.616 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:43:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:52.616 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 36, 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 172650, 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:52.617 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:52.618 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:43:52.619 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:43:52.621 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:43:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:43:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:43:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:52.622 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:43:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:43:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.623 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.624 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:52.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:52.626 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:43:52.627 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.628 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.629 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x2f018fb87aa4aafe", 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:52.630 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:43:52.631 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:43:52.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92e3000640 1 -- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f92bc06be30 msgr2=0x7f92bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92e3000640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f92bc06be30 0x7f92bc06e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f92d4034000 tx=0x7f92d402e000 comp rx=0 tx=0).stop 2024-04-24T22:43:52.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92e3000640 1 -- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92dc0c67e0 msgr2=0x7f92dc156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92e3000640 1 --2- 172.21.15.47:0/2815561543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92dc0c67e0 0x7f92dc156fa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00c430 tx=0x7f92cc00c8f0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92e2600640 1 -- 172.21.15.47:0/2815561543 reap_dead start 2024-04-24T22:43:52.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92e2600640 1 -- 172.21.15.47:0/2815561543 reap_dead start 2024-04-24T22:43:52.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92e3000640 1 -- 172.21.15.47:0/2815561543 shutdown_connections 2024-04-24T22:43:52.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92e3000640 1 -- 172.21.15.47:0/2815561543 >> 172.21.15.47:0/2815561543 conn(0x7f92dc0c9410 msgr2=0x7f92dc0c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:52.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92e3000640 1 -- 172.21.15.47:0/2815561543 shutdown_connections 2024-04-24T22:43:52.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.613+0000 7f92e3000640 1 -- 172.21.15.47:0/2815561543 wait complete. 2024-04-24T22:43:52.634 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:43:52.634 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-24T22:43:52.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-24T22:43:52.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.834+0000 7fbb87e00640 1 Processor -- start 2024-04-24T22:43:52.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.834+0000 7fbb87e00640 1 -- start start 2024-04-24T22:43:52.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.836+0000 7fbb87e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb88072c60 0x7fbb880c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.836+0000 7fbb87e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb8815c3d0 con 0x7fbb88073040 2024-04-24T22:43:52.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.836+0000 7fbb87e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb8815c6b0 con 0x7fbb8806bb90 2024-04-24T22:43:52.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.836+0000 7fbb87e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbb88158bf0 con 0x7fbb88072c60 2024-04-24T22:43:52.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.836+0000 7fbb86000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbb88073040 0x7fbb881411f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37350/0 (socket says 172.21.15.47:37350) 2024-04-24T22:43:52.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.836+0000 7fbb87400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbb8806bb90 0x7fbb8815b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:42902/0 (socket says 172.21.15.47:42902) 2024-04-24T22:43:52.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.836+0000 7fbb87400640 1 -- 172.21.15.47:0/1450442135 learned_addr learned my addr 172.21.15.47:0/1450442135 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:43:52.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.836+0000 7fbb86a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb88072c60 0x7fbb880c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.837+0000 7fbb85600640 1 -- 172.21.15.47:0/1450442135 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 850940414 0 0) 0x7fbb8815c6b0 con 0x7fbb8806bb90 2024-04-24T22:43:52.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.837+0000 7fbb86a00640 1 -- 172.21.15.47:0/1450442135 >> v1:172.21.15.113:6789/0 conn(0x7fbb8806bb90 legacy=0x7fbb8815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.837+0000 7fbb86a00640 1 -- 172.21.15.47:0/1450442135 >> v1:172.21.15.47:6789/0 conn(0x7fbb88073040 legacy=0x7fbb881411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.837+0000 7fbb86a00640 1 -- 172.21.15.47:0/1450442135 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb881585a0 con 0x7fbb88072c60 2024-04-24T22:43:52.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.838+0000 7fbb86a00640 1 --2- 172.21.15.47:0/1450442135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb88072c60 0x7fbb880c2d60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb780097d0 tx=0x7fbb7802a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f14279534150963f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.838+0000 7fbb85600640 1 -- 172.21.15.47:0/1450442135 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78037070 con 0x7fbb88072c60 2024-04-24T22:43:52.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.838+0000 7fbb85600640 1 -- 172.21.15.47:0/1450442135 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb78032440 con 0x7fbb88072c60 2024-04-24T22:43:52.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.838+0000 7fbb85600640 1 -- 172.21.15.47:0/1450442135 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78032810 con 0x7fbb88072c60 2024-04-24T22:43:52.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.839+0000 7fbb87e00640 1 -- 172.21.15.47:0/1450442135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb88072c60 msgr2=0x7fbb880c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.839+0000 7fbb87e00640 1 --2- 172.21.15.47:0/1450442135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb88072c60 0x7fbb880c2d60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb780097d0 tx=0x7fbb7802a2a0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.839+0000 7fbb87e00640 1 -- 172.21.15.47:0/1450442135 shutdown_connections 2024-04-24T22:43:52.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.839+0000 7fbb87e00640 1 --2- 172.21.15.47:0/1450442135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb88072c60 0x7fbb880c2d60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.839+0000 7fbb87e00640 1 -- 172.21.15.47:0/1450442135 >> 172.21.15.47:0/1450442135 conn(0x7fbb880cb900 msgr2=0x7fbb880cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:52.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.839+0000 7fbb87e00640 1 -- 172.21.15.47:0/1450442135 shutdown_connections 2024-04-24T22:43:52.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.839+0000 7fbb87e00640 1 -- 172.21.15.47:0/1450442135 wait complete. 2024-04-24T22:43:52.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.840+0000 7fbb87e00640 1 Processor -- start 2024-04-24T22:43:52.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.841+0000 7fbb87e00640 1 -- start start 2024-04-24T22:43:52.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.841+0000 7fbb87e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb8806bb90 0x7fbb880c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88072c60 0x7fbb880c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb88073040 0x7fbb880c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbb880c32a0 con 0x7fbb88073040 2024-04-24T22:43:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb88158270 con 0x7fbb88072c60 2024-04-24T22:43:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbb88159080 con 0x7fbb8806bb90 2024-04-24T22:43:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb88073040 0x7fbb880c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb88073040 0x7fbb880c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48160/0 (socket says 172.21.15.47:48160) 2024-04-24T22:43:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87400640 1 -- 172.21.15.47:0/875844880 learned_addr learned my addr 172.21.15.47:0/875844880 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb86a00640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb8806bb90 0x7fbb880c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb86000640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88072c60 0x7fbb880c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87400640 1 -- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb8806bb90 msgr2=0x7fbb880c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87400640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb8806bb90 0x7fbb880c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87400640 1 -- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88072c60 msgr2=0x7fbb880c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87400640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88072c60 0x7fbb880c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.842+0000 7fbb87400640 1 -- 172.21.15.47:0/875844880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb880c6f30 con 0x7fbb88073040 2024-04-24T22:43:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.843+0000 7fbb87400640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb88073040 0x7fbb880c81e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c7d0 tx=0x7fbb8000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.843+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80018070 con 0x7fbb88073040 2024-04-24T22:43:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.843+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8000a640 con 0x7fbb88073040 2024-04-24T22:43:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.843+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000aa70 con 0x7fbb88073040 2024-04-24T22:43:52.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.843+0000 7fbb87e00640 1 -- 172.21.15.47:0/875844880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb880c92f0 con 0x7fbb88073040 2024-04-24T22:43:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.843+0000 7fbb87e00640 1 -- 172.21.15.47:0/875844880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb880c69b0 con 0x7fbb88073040 2024-04-24T22:43:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.844+0000 7fbb87e00640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb880c76e0 0x7fbb88155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.844+0000 7fbb87e00640 1 -- 172.21.15.47:0/875844880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb881f0f60 con 0x7fbb880c76e0 2024-04-24T22:43:52.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.845+0000 7fbb86a00640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb880c76e0 0x7fbb88155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.845+0000 7fbb86a00640 1 -- 172.21.15.47:0/875844880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb88068d50 con 0x7fbb880c76e0 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.845+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb8000ac10 con 0x7fbb88073040 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.845+0000 7fbb86a00640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb880c76e0 0x7fbb88155050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbb880cca00 tx=0x7fbb780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.846+0000 7fbb7f400640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbb6806be30 0x7fbb6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.846+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbb8008a5e0 con 0x7fbb88073040 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.846+0000 7fbb86000640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbb6806be30 0x7fbb6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.847+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78037040 con 0x7fbb880c76e0 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.847+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbb7803c640 con 0x7fbb880c76e0 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.847+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb880c76e0 msgr2=0x7fbb88155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.847+0000 7fbb7f400640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb880c76e0 0x7fbb88155050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbb880cca00 tx=0x7fbb780046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.847+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb880c76e0 msgr2=0x7fbb88155050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.847+0000 7fbb7f400640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb880c76e0 0x7fbb88155050 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbb880cca00 tx=0x7fbb780046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.847+0000 7fbb86000640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbb6806be30 0x7fbb6806e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbb881edbb0 tx=0x7fbb74000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.854+0000 7fbb87e00640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88155590 0x7fbb8813e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.854+0000 7fbb87e00640 1 -- 172.21.15.47:0/875844880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbb881f0ca0 con 0x7fbb88155590 2024-04-24T22:43:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.855+0000 7fbb86a00640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88155590 0x7fbb8813e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.855+0000 7fbb86a00640 1 -- 172.21.15.47:0/875844880 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbb88152940 con 0x7fbb88155590 2024-04-24T22:43:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.855+0000 7fbb86a00640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88155590 0x7fbb8813e0c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbb78000c00 tx=0x7fbb78008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.856+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7804c070 con 0x7fbb88155590 2024-04-24T22:43:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.856+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 <== mon.1 v2:172.21.15.113:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4553 (secure 0 0 0) 0x7fbb7803bce0 con 0x7fbb88155590 2024-04-24T22:43:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.856+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88155590 msgr2=0x7fbb8813e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.856+0000 7fbb7f400640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88155590 0x7fbb8813e0c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbb78000c00 tx=0x7fbb78008030 comp rx=0 tx=0).stop 2024-04-24T22:43:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.856+0000 7fbb7f400640 1 -- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88155590 msgr2=0x7fbb8813e0c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.856+0000 7fbb7f400640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb88155590 0x7fbb8813e0c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.859 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:43:52.859 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:43:52.859 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 36, 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 8134, 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:52.860 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.861 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:52.862 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:43:52.863 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:52.864 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:43:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:43:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:52.865 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.867 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:43:52.867 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:43:52.867 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:52.867 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:52.867 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:52.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.867 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.867 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:43:52.867 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.869 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:43:52.869 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:43:52.869 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:52.869 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:52.869 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:52.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.869 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.869 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:52.869 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:52.869 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:43:52.871 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:52.872 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:52.873 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:52.873 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:52.873 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:43:52.873 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:43:52.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.856+0000 7fbb87e00640 1 -- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbb6806be30 msgr2=0x7fbb6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.856+0000 7fbb87e00640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fbb6806be30 0x7fbb6806e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbb881edbb0 tx=0x7fbb74000ed0 comp rx=0 tx=0).stop 2024-04-24T22:43:52.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.857+0000 7fbb87e00640 1 -- 172.21.15.47:0/875844880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb88073040 msgr2=0x7fbb880c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:52.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.857+0000 7fbb87e00640 1 --2- 172.21.15.47:0/875844880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb88073040 0x7fbb880c81e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c7d0 tx=0x7fbb8000cc90 comp rx=0 tx=0).stop 2024-04-24T22:43:52.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.857+0000 7fbb87400640 1 -- 172.21.15.47:0/875844880 reap_dead start 2024-04-24T22:43:52.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.857+0000 7fbb87400640 1 -- 172.21.15.47:0/875844880 reap_dead start 2024-04-24T22:43:52.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.857+0000 7fbb87e00640 1 -- 172.21.15.47:0/875844880 shutdown_connections 2024-04-24T22:43:52.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.857+0000 7fbb87e00640 1 -- 172.21.15.47:0/875844880 >> 172.21.15.47:0/875844880 conn(0x7fbb880cb900 msgr2=0x7fbb880c1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:52.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.857+0000 7fbb87e00640 1 -- 172.21.15.47:0/875844880 shutdown_connections 2024-04-24T22:43:52.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:52.857+0000 7fbb87e00640 1 -- 172.21.15.47:0/875844880 wait complete. 2024-04-24T22:43:52.878 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:43:52.878 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-24T22:43:52.878 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-24T22:43:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.094+0000 7fc1c0800640 1 Processor -- start 2024-04-24T22:43:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.095+0000 7fc1c0800640 1 -- start start 2024-04-24T22:43:53.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.096+0000 7fc1c0800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1b80cc840 0x7fc1b80ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.096+0000 7fc1c0800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cbde0 0x7fc1b80ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.096+0000 7fc1c0800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1b815b240 con 0x7fc1b8073380 2024-04-24T22:43:53.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.096+0000 7fc1c0800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b815daa0 con 0x7fc1b80cc840 2024-04-24T22:43:53.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.096+0000 7fc1c0800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b815dcf0 con 0x7fc1b80cbde0 2024-04-24T22:43:53.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.097+0000 7fc1bde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1b80cc840 0x7fc1b80ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.097+0000 7fc1be800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cbde0 0x7fc1b80ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.097+0000 7fc1bde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1b80cc840 0x7fc1b80ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:50412/0 (socket says 172.21.15.47:50412) 2024-04-24T22:43:53.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.097+0000 7fc1bde00640 1 -- 172.21.15.47:0/1291522645 learned_addr learned my addr 172.21.15.47:0/1291522645 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:53.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.097+0000 7fc1bd400640 1 --1- 172.21.15.47:0/1291522645 >> v1:172.21.15.113:6789/0 conn(0x7fc1b8073380 0x7fc1b80cb6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:42906/0 (socket says 172.21.15.47:42906) 2024-04-24T22:43:53.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.097+0000 7fc1bde00640 1 -- 172.21.15.47:0/1291522645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cbde0 msgr2=0x7fc1b80ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:53.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.097+0000 7fc1bde00640 1 --2- 172.21.15.47:0/1291522645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cbde0 0x7fc1b80ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.097+0000 7fc1bde00640 1 -- 172.21.15.47:0/1291522645 >> v1:172.21.15.113:6789/0 conn(0x7fc1b8073380 legacy=0x7fc1b80cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.097+0000 7fc1bde00640 1 -- 172.21.15.47:0/1291522645 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b815dfd0 con 0x7fc1b80cc840 2024-04-24T22:43:53.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.098+0000 7fc1bde00640 1 --2- 172.21.15.47:0/1291522645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b80cc840 0x7fc1b80ca7d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009e50 tx=0x7fc1b002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e238de50fe377099 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:53.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.099+0000 7fc1bca00640 1 -- 172.21.15.47:0/1291522645 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0036070 con 0x7fc1b80cc840 2024-04-24T22:43:53.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.099+0000 7fc1bca00640 1 -- 172.21.15.47:0/1291522645 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b0031600 con 0x7fc1b80cc840 2024-04-24T22:43:53.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.099+0000 7fc1bca00640 1 -- 172.21.15.47:0/1291522645 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00319d0 con 0x7fc1b80cc840 2024-04-24T22:43:53.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.099+0000 7fc1c0800640 1 -- 172.21.15.47:0/1291522645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b80cc840 msgr2=0x7fc1b80ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.099+0000 7fc1c0800640 1 --2- 172.21.15.47:0/1291522645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b80cc840 0x7fc1b80ca7d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009e50 tx=0x7fc1b002a710 comp rx=0 tx=0).stop 2024-04-24T22:43:53.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.100+0000 7fc1c0800640 1 -- 172.21.15.47:0/1291522645 shutdown_connections 2024-04-24T22:43:53.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.100+0000 7fc1c0800640 1 --2- 172.21.15.47:0/1291522645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cbde0 0x7fc1b80ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.100+0000 7fc1c0800640 1 --2- 172.21.15.47:0/1291522645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b80cc840 0x7fc1b80ca7d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.100+0000 7fc1c0800640 1 -- 172.21.15.47:0/1291522645 >> 172.21.15.47:0/1291522645 conn(0x7fc1b806c940 msgr2=0x7fc1b8072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:53.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.100+0000 7fc1c0800640 1 -- 172.21.15.47:0/1291522645 shutdown_connections 2024-04-24T22:43:53.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.100+0000 7fc1c0800640 1 -- 172.21.15.47:0/1291522645 wait complete. 2024-04-24T22:43:53.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.102+0000 7fc1c0800640 1 Processor -- start 2024-04-24T22:43:53.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.102+0000 7fc1c0800640 1 -- start start 2024-04-24T22:43:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1c0800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b8073380 0x7fc1b80d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1c0800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1b80cbde0 0x7fc1b81502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1c0800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cc840 0x7fc1b80ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1c0800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b815e870 con 0x7fc1b80cbde0 2024-04-24T22:43:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1c0800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b815e590 con 0x7fc1b8073380 2024-04-24T22:43:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1c0800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b815af30 con 0x7fc1b80cc840 2024-04-24T22:43:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1bde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b8073380 0x7fc1b80d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1bd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1b80cbde0 0x7fc1b81502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1bde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b8073380 0x7fc1b80d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:50416/0 (socket says 172.21.15.47:50416) 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1bde00640 1 -- 172.21.15.47:0/2213176365 learned_addr learned my addr 172.21.15.47:0/2213176365 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.103+0000 7fc1be800640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cc840 0x7fc1b80ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.104+0000 7fc1bde00640 1 -- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cc840 msgr2=0x7fc1b80ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.104+0000 7fc1bde00640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cc840 0x7fc1b80ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.104+0000 7fc1bde00640 1 -- 172.21.15.47:0/2213176365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1b80cbde0 msgr2=0x7fc1b81502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.104+0000 7fc1bde00640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1b80cbde0 0x7fc1b81502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.104+0000 7fc1bde00640 1 -- 172.21.15.47:0/2213176365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b80c7070 con 0x7fc1b8073380 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.104+0000 7fc1bde00640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b8073380 0x7fc1b80d00a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009f70 tx=0x7fc1b0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.104+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b003f020 con 0x7fc1b8073380 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.105+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b0037040 con 0x7fc1b8073380 2024-04-24T22:43:53.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.105+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0036070 con 0x7fc1b8073380 2024-04-24T22:43:53.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.105+0000 7fc1c0800640 1 -- 172.21.15.47:0/2213176365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b81e4890 con 0x7fc1b8073380 2024-04-24T22:43:53.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.105+0000 7fc1c0800640 1 -- 172.21.15.47:0/2213176365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b81e4e50 con 0x7fc1b8073380 2024-04-24T22:43:53.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.106+0000 7fc1c0800640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80ce6b0 0x7fc1b8145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.106+0000 7fc1c0800640 1 -- 172.21.15.47:0/2213176365 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b80b5270 con 0x7fc1b80ce6b0 2024-04-24T22:43:53.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.106+0000 7fc1bd400640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80ce6b0 0x7fc1b8145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.106+0000 7fc1bd400640 1 -- 172.21.15.47:0/2213176365 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1b8073b90 con 0x7fc1b80ce6b0 2024-04-24T22:43:53.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.107+0000 7fc1bd400640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80ce6b0 0x7fc1b8145280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80c5a80 tx=0x7fc1a8001880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:53.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.107+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1b0042020 con 0x7fc1b8073380 2024-04-24T22:43:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.108+0000 7fc1ae000640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc19c06be30 0x7fc19c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.108+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8011580 con 0x7fc1b80ce6b0 2024-04-24T22:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.108+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc1b00ab280 con 0x7fc1b8073380 2024-04-24T22:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.108+0000 7fc1be800640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc19c06be30 0x7fc19c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.108+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc1a8011720 con 0x7fc1b80ce6b0 2024-04-24T22:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.108+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80ce6b0 msgr2=0x7fc1b8145280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.108+0000 7fc1ae000640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80ce6b0 0x7fc1b8145280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80c5a80 tx=0x7fc1a8001880 comp rx=0 tx=0).stop 2024-04-24T22:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.108+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80ce6b0 msgr2=0x7fc1b8145280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.108+0000 7fc1ae000640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80ce6b0 0x7fc1b8145280 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80c5a80 tx=0x7fc1a8001880 comp rx=0 tx=0).stop 2024-04-24T22:43:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.109+0000 7fc1be800640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc19c06be30 0x7fc19c06e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80ca9b0 tx=0x7fc1b4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:53.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.116+0000 7fc1c0800640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cf0b0 0x7fc1b81457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.116+0000 7fc1c0800640 1 -- 172.21.15.47:0/2213176365 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b80cadb0 con 0x7fc1b80cf0b0 2024-04-24T22:43:53.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.116+0000 7fc1bd400640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cf0b0 0x7fc1b81457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.116+0000 7fc1bd400640 1 -- 172.21.15.47:0/2213176365 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc1b80ca280 con 0x7fc1b80cf0b0 2024-04-24T22:43:53.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.117+0000 7fc1bd400640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cf0b0 0x7fc1b81457c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8014000 tx=0x7fc1a8000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:53.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.117+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a801e110 con 0x7fc1b80cf0b0 2024-04-24T22:43:53.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.117+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 <== mon.2 v2:172.21.15.113:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5064 (secure 0 0 0) 0x7fc1a801e110 con 0x7fc1b80cf0b0 2024-04-24T22:43:53.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.117+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cf0b0 msgr2=0x7fc1b81457c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.117+0000 7fc1ae000640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cf0b0 0x7fc1b81457c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8014000 tx=0x7fc1a8000ed0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.117+0000 7fc1ae000640 1 -- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cf0b0 msgr2=0x7fc1b81457c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-24T22:43:53.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.117+0000 7fc1ae000640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b80cf0b0 0x7fc1b81457c0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8014000 tx=0x7fc1a8000ed0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 36, 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 66759, 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: 2 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:53.121 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-24T22:43:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2450521708007866372", 2024-04-24T22:43:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-24T22:43:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:53.122 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:53.123 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540701547738038271", 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:53.124 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "617410c3-1ae2-44fe-8492-d21ee235c3cc", 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-24T22:38:41.976658Z", 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 19, 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "squid", 2024-04-24T22:43:53.125 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 1, 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-24T22:43:53.126 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "reef", 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "squid" 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-24T22:43:53.128 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3300", 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789", 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6789/0", 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6789/0", 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:53.130 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:3301", 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-24T22:43:53.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790", 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.113:6790/0", 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.113:6790/0", 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-24T22:43:53.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": [ 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:43:53.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:53.134 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-24T22:43:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 7 2024-04-24T22:43:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:53.135 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-24T22:43:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-24T22:43:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:53.136 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-24T22:43:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-24T22:43:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-24T22:43:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f03cffffffdffff", 2024-04-24T22:43:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-24T22:43:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-24T22:43:53.137 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-24T22:43:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-24T22:43:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-24T22:43:53.138 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-24T22:43:53.138 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:43:53.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.118+0000 7fc1c0800640 1 -- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc19c06be30 msgr2=0x7fc19c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.118+0000 7fc1c0800640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc19c06be30 0x7fc19c06e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80ca9b0 tx=0x7fc1b4009190 comp rx=0 tx=0).stop 2024-04-24T22:43:53.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.118+0000 7fc1c0800640 1 -- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b8073380 msgr2=0x7fc1b80d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.118+0000 7fc1c0800640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b8073380 0x7fc1b80d00a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009f70 tx=0x7fc1b0004a40 comp rx=0 tx=0).stop 2024-04-24T22:43:53.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.118+0000 7fc1be800640 1 -- 172.21.15.47:0/2213176365 reap_dead start 2024-04-24T22:43:53.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.118+0000 7fc1c0800640 1 -- 172.21.15.47:0/2213176365 shutdown_connections 2024-04-24T22:43:53.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.118+0000 7fc1c0800640 1 --2- 172.21.15.47:0/2213176365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b8073380 0x7fc1b80d00a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.118+0000 7fc1c0800640 1 -- 172.21.15.47:0/2213176365 >> 172.21.15.47:0/2213176365 conn(0x7fc1b806c940 msgr2=0x7fc1b8153aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:53.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.118+0000 7fc1c0800640 1 -- 172.21.15.47:0/2213176365 shutdown_connections 2024-04-24T22:43:53.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.118+0000 7fc1c0800640 1 -- 172.21.15.47:0/2213176365 wait complete. 2024-04-24T22:43:53.143 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-24T22:43:53.143 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-24T22:43:53.144 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-24T22:43:53.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.357+0000 7fd5c1a00640 1 Processor -- start 2024-04-24T22:43:53.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.357+0000 7fd5c1a00640 1 -- start start 2024-04-24T22:43:53.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.359+0000 7fd5c1a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5bc152960 0x7fd5bc0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.359+0000 7fd5c1a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5bc072b80 0x7fd5bc072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.359+0000 7fd5c1a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5bc0c9400 con 0x7fd5bc0c8fa0 2024-04-24T22:43:53.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.359+0000 7fd5c1a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd5bc158f20 con 0x7fd5bc072b80 2024-04-24T22:43:53.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.359+0000 7fd5c1a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd5bc1593b0 con 0x7fd5bc152960 2024-04-24T22:43:53.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.359+0000 7fd5bb400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5bc072b80 0x7fd5bc072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.359+0000 7fd5bb400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5bc072b80 0x7fd5bc072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48172/0 (socket says 172.21.15.47:48172) 2024-04-24T22:43:53.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.360+0000 7fd5bb400640 1 -- 172.21.15.47:0/2592330956 learned_addr learned my addr 172.21.15.47:0/2592330956 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:53.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.360+0000 7fd5baa00640 1 --2- 172.21.15.47:0/2592330956 >> v2:172.21.15.113:3300/0 conn(0x7fd5bc152960 0x7fd5bc0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.360+0000 7fd5baa00640 1 -- 172.21.15.47:0/2592330956 >> v2:172.21.15.47:3300/0 conn(0x7fd5bc072b80 msgr2=0x7fd5bc072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:43:53.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.360+0000 7fd5baa00640 1 --2- 172.21.15.47:0/2592330956 >> v2:172.21.15.47:3300/0 conn(0x7fd5bc072b80 0x7fd5bc072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.360+0000 7fd5baa00640 1 -- 172.21.15.47:0/2592330956 >> v1:172.21.15.113:6789/0 conn(0x7fd5bc0c8fa0 legacy=0x7fd5bc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.360+0000 7fd5baa00640 1 -- 172.21.15.47:0/2592330956 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc0c7e50 con 0x7fd5bc152960 2024-04-24T22:43:53.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.361+0000 7fd5baa00640 1 --2- 172.21.15.47:0/2592330956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc152960 0x7fd5bc0c7750 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc070600 tx=0x7fd5ac02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d96e7492634c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:53.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.361+0000 7fd5ba000640 1 -- 172.21.15.47:0/2592330956 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac037070 con 0x7fd5bc152960 2024-04-24T22:43:53.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.361+0000 7fd5ba000640 1 -- 172.21.15.47:0/2592330956 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5ac032440 con 0x7fd5bc152960 2024-04-24T22:43:53.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.361+0000 7fd5ba000640 1 -- 172.21.15.47:0/2592330956 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac032810 con 0x7fd5bc152960 2024-04-24T22:43:53.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.362+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2592330956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc152960 msgr2=0x7fd5bc0c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.362+0000 7fd5c1a00640 1 --2- 172.21.15.47:0/2592330956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc152960 0x7fd5bc0c7750 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc070600 tx=0x7fd5ac02a250 comp rx=0 tx=0).stop 2024-04-24T22:43:53.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.362+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2592330956 shutdown_connections 2024-04-24T22:43:53.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.362+0000 7fd5c1a00640 1 --2- 172.21.15.47:0/2592330956 >> v2:172.21.15.47:3300/0 conn(0x7fd5bc072b80 0x7fd5bc072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.362+0000 7fd5c1a00640 1 --2- 172.21.15.47:0/2592330956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc152960 0x7fd5bc0c7750 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.362+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2592330956 >> 172.21.15.47:0/2592330956 conn(0x7fd5bc06e580 msgr2=0x7fd5bc06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:53.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.362+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2592330956 shutdown_connections 2024-04-24T22:43:53.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.362+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2592330956 wait complete. 2024-04-24T22:43:53.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.364+0000 7fd5c1a00640 1 Processor -- start 2024-04-24T22:43:53.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.364+0000 7fd5c1a00640 1 -- start start 2024-04-24T22:43:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.365+0000 7fd5c1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc072b80 0x7fd5bc0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.365+0000 7fd5c1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0c8fa0 0x7fd5bc0b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.365+0000 7fd5c1a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc152960 0x7fd5bc0c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.365+0000 7fd5c1a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc1588d0 con 0x7fd5bc0c8fa0 2024-04-24T22:43:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.365+0000 7fd5c1a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc0c8130 con 0x7fd5bc072b80 2024-04-24T22:43:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.365+0000 7fd5c1a00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd5bc158bb0 con 0x7fd5bc152960 2024-04-24T22:43:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5b3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0c8fa0 0x7fd5bc0b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5baa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc072b80 0x7fd5bc0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5baa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc072b80 0x7fd5bc0b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:50434/0 (socket says 172.21.15.47:50434) 2024-04-24T22:43:53.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5baa00640 1 -- 172.21.15.47:0/2099643172 learned_addr learned my addr 172.21.15.47:0/2099643172 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:43:53.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5bb400640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc152960 0x7fd5bc0c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5baa00640 1 -- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc152960 msgr2=0x7fd5bc0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5baa00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc152960 0x7fd5bc0c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5baa00640 1 -- 172.21.15.47:0/2099643172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0c8fa0 msgr2=0x7fd5bc0b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5baa00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0c8fa0 0x7fd5bc0b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5baa00640 1 -- 172.21.15.47:0/2099643172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc0c5800 con 0x7fd5bc072b80 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.366+0000 7fd5baa00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc072b80 0x7fd5bc0b0fd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac000c00 tx=0x7fd5ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.367+0000 7fd5b2a00640 1 -- 172.21.15.47:0/2099643172 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac037040 con 0x7fd5bc072b80 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.367+0000 7fd5b2a00640 1 -- 172.21.15.47:0/2099643172 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5ac042070 con 0x7fd5bc072b80 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.367+0000 7fd5b2a00640 1 -- 172.21.15.47:0/2099643172 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac030960 con 0x7fd5bc072b80 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.367+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2099643172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc0b1610 con 0x7fd5bc072b80 2024-04-24T22:43:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.367+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2099643172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc1f07e0 con 0x7fd5bc072b80 2024-04-24T22:43:53.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.368+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2099643172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5bc073b90 con 0x7fd5bc072b80 2024-04-24T22:43:53.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.369+0000 7fd5b2a00640 1 -- 172.21.15.47:0/2099643172 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd5ac030b00 con 0x7fd5bc072b80 2024-04-24T22:43:53.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.370+0000 7fd5b2a00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd59806be80 0x7fd59806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:43:53.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.370+0000 7fd5b2a00640 1 -- 172.21.15.47:0/2099643172 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd5ac0ab750 con 0x7fd5bc072b80 2024-04-24T22:43:53.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.370+0000 7fd5b3e00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd59806be80 0x7fd59806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:43:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.371+0000 7fd5b3e00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd59806be80 0x7fd59806e340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc14f700 tx=0x7fd5a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:43:53.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.374+0000 7fd5b2a00640 1 -- 172.21.15.47:0/2099643172 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd5ac079a10 con 0x7fd5bc072b80 2024-04-24T22:43:53.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.653+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2099643172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd5bc06c200 con 0x7fd5bc072b80 2024-04-24T22:43:53.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.656+0000 7fd5b2a00640 1 -- 172.21.15.47:0/2099643172 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fd5ac07dd10 con 0x7fd5bc072b80 2024-04-24T22:43:53.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd59806be80 msgr2=0x7fd59806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd59806be80 0x7fd59806e340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc14f700 tx=0x7fd5a4009040 comp rx=0 tx=0).stop 2024-04-24T22:43:53.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc072b80 msgr2=0x7fd5bc0b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:43:53.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc072b80 0x7fd5bc0b0fd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac000c00 tx=0x7fd5ac0046c0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2099643172 shutdown_connections 2024-04-24T22:43:53.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd59806be80 0x7fd59806e340 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5bc152960 0x7fd5bc0c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc0c8fa0 0x7fd5bc0b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 --2- 172.21.15.47:0/2099643172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5bc072b80 0x7fd5bc0b0fd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:43:53.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2099643172 >> 172.21.15.47:0/2099643172 conn(0x7fd5bc06e580 msgr2=0x7fd5bc151910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:43:53.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2099643172 shutdown_connections 2024-04-24T22:43:53.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:43:53.659+0000 7fd5c1a00640 1 -- 172.21.15.47:0/2099643172 wait complete. 2024-04-24T22:43:53.683 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-24T22:44:03.684 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:44:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.904+0000 7f0e0a800640 1 Processor -- start 2024-04-24T22:44:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.904+0000 7f0e0a800640 1 -- start start 2024-04-24T22:44:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.906+0000 7f0e0a800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e041514f0 0x7f0e0406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.906+0000 7f0e0a800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e040c6a20 0x7f0e040707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.906+0000 7f0e0a800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e04071570 con 0x7f0e0406e6f0 2024-04-24T22:44:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.906+0000 7f0e0a800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f0e0415b470 con 0x7f0e040c6a20 2024-04-24T22:44:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.906+0000 7f0e0a800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0e0415a8a0 con 0x7f0e041514f0 2024-04-24T22:44:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.906+0000 7f0e09e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e040c6a20 0x7f0e040707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.906+0000 7f0e09e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0e040c6a20 0x7f0e040707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51194/0 (socket says 172.21.15.47:51194) 2024-04-24T22:44:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.906+0000 7f0e09400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e041514f0 0x7f0e0406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.906+0000 7f0e09e00640 1 -- 172.21.15.47:0/562870165 learned_addr learned my addr 172.21.15.47:0/562870165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:03.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.907+0000 7f0e09400640 1 -- 172.21.15.47:0/562870165 >> v2:172.21.15.47:3300/0 conn(0x7f0e040c6a20 msgr2=0x7f0e040707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:03.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.907+0000 7f0e09400640 1 --2- 172.21.15.47:0/562870165 >> v2:172.21.15.47:3300/0 conn(0x7f0e040c6a20 0x7f0e040707e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:03.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.907+0000 7f0e09400640 1 -- 172.21.15.47:0/562870165 >> v1:172.21.15.113:6789/0 conn(0x7f0e0406e6f0 legacy=0x7f0e040c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:03.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.907+0000 7f0e09400640 1 -- 172.21.15.47:0/562870165 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e041518d0 con 0x7f0e041514f0 2024-04-24T22:44:03.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.908+0000 7f0e09400640 1 --2- 172.21.15.47:0/562870165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e041514f0 0x7f0e0406e1b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0e0406c380 tx=0x7f0df802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe267f2caa38a0d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:03.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.908+0000 7f0e03e00640 1 -- 172.21.15.47:0/562870165 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8037070 con 0x7f0e041514f0 2024-04-24T22:44:03.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.908+0000 7f0e03e00640 1 -- 172.21.15.47:0/562870165 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8032440 con 0x7f0e041514f0 2024-04-24T22:44:03.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.908+0000 7f0e03e00640 1 -- 172.21.15.47:0/562870165 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8032810 con 0x7f0e041514f0 2024-04-24T22:44:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.909+0000 7f0e0a800640 1 -- 172.21.15.47:0/562870165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e041514f0 msgr2=0x7f0e0406e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.909+0000 7f0e0a800640 1 --2- 172.21.15.47:0/562870165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e041514f0 0x7f0e0406e1b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0e0406c380 tx=0x7f0df802a1a0 comp rx=0 tx=0).stop 2024-04-24T22:44:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.909+0000 7f0e0a800640 1 -- 172.21.15.47:0/562870165 shutdown_connections 2024-04-24T22:44:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.909+0000 7f0e0a800640 1 --2- 172.21.15.47:0/562870165 >> v2:172.21.15.47:3300/0 conn(0x7f0e040c6a20 0x7f0e040707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.909+0000 7f0e0a800640 1 --2- 172.21.15.47:0/562870165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e041514f0 0x7f0e0406e1b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.909+0000 7f0e0a800640 1 -- 172.21.15.47:0/562870165 >> 172.21.15.47:0/562870165 conn(0x7f0e040cb900 msgr2=0x7f0e040cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.909+0000 7f0e0a800640 1 -- 172.21.15.47:0/562870165 shutdown_connections 2024-04-24T22:44:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.909+0000 7f0e0a800640 1 -- 172.21.15.47:0/562870165 wait complete. 2024-04-24T22:44:03.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.911+0000 7f0e0a800640 1 Processor -- start 2024-04-24T22:44:03.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.911+0000 7f0e0a800640 1 -- start start 2024-04-24T22:44:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e0a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0406e6f0 0x7f0e04152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e0a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e040c6a20 0x7f0e041e0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e0a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e041514f0 0x7f0e040c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e0a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0415b100 con 0x7f0e040c6a20 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e0a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0415ae20 con 0x7f0e0406e6f0 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e0a800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e040c7ba0 con 0x7f0e041514f0 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e08a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e040c6a20 0x7f0e041e0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e09400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0406e6f0 0x7f0e04152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e08a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e040c6a20 0x7f0e041e0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51200/0 (socket says 172.21.15.47:51200) 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e08a00640 1 -- 172.21.15.47:0/4086248598 learned_addr learned my addr 172.21.15.47:0/4086248598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.912+0000 7f0e09e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e041514f0 0x7f0e040c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e09400640 1 -- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e041514f0 msgr2=0x7f0e040c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e09400640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e041514f0 0x7f0e040c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e09400640 1 -- 172.21.15.47:0/4086248598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e040c6a20 msgr2=0x7f0e041e0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e09400640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e040c6a20 0x7f0e041e0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e09400640 1 -- 172.21.15.47:0/4086248598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e04153ed0 con 0x7f0e0406e6f0 2024-04-24T22:44:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e08a00640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e040c6a20 0x7f0e041e0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:03.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e09400640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0406e6f0 0x7f0e04152520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0e0415b5f0 tx=0x7f0df80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:03.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e01600640 1 -- 172.21.15.47:0/4086248598 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8041030 con 0x7f0e0406e6f0 2024-04-24T22:44:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e01600640 1 -- 172.21.15.47:0/4086248598 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8030500 con 0x7f0e0406e6f0 2024-04-24T22:44:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e01600640 1 -- 172.21.15.47:0/4086248598 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8037040 con 0x7f0e0406e6f0 2024-04-24T22:44:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.913+0000 7f0e0a800640 1 -- 172.21.15.47:0/4086248598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e04154170 con 0x7f0e0406e6f0 2024-04-24T22:44:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.914+0000 7f0e0a800640 1 -- 172.21.15.47:0/4086248598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e04154730 con 0x7f0e0406e6f0 2024-04-24T22:44:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.915+0000 7f0e0a800640 1 -- 172.21.15.47:0/4086248598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e040c2e70 con 0x7f0e0406e6f0 2024-04-24T22:44:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.916+0000 7f0e01600640 1 -- 172.21.15.47:0/4086248598 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0df8044070 con 0x7f0e0406e6f0 2024-04-24T22:44:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.917+0000 7f0e01600640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0de406be30 0x7f0de406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.917+0000 7f0e01600640 1 -- 172.21.15.47:0/4086248598 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0df80ab460 con 0x7f0e0406e6f0 2024-04-24T22:44:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.917+0000 7f0e08a00640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0de406be30 0x7f0de406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.918+0000 7f0e08a00640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0de406be30 0x7f0de406e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0e041ede50 tx=0x7f0df0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:03.922+0000 7f0e01600640 1 -- 172.21.15.47:0/4086248598 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0e040c2e70 con 0x7f0e0406e6f0 2024-04-24T22:44:04.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.166+0000 7f0e0a800640 1 -- 172.21.15.47:0/4086248598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e04068d50 con 0x7f0e0406e6f0 2024-04-24T22:44:04.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.168+0000 7f0e01600640 1 -- 172.21.15.47:0/4086248598 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6916 (secure 0 0 0) 0x7f0df8079850 con 0x7f0e0406e6f0 2024-04-24T22:44:04.171 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:04.171 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1712577517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1712577517},{"type":"v1","addr":"172.21.15.113:6839","nonce":1712577517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:39:37.178621+0000","modified":"2024-04-24T22:40:06.837213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4282,"mds_2":4279,"mds_3":4241,"mds_4":4238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4238":{"gid":4238,"name":"a","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6833/2908847255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2908847255},{"type":"v1","addr":"172.21.15.47:6833","nonce":2908847255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.47:6837/2173854280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2173854280},{"type":"v1","addr":"172.21.15.47:6837","nonce":2173854280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/3103107210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3103107210},{"type":"v1","addr":"172.21.15.113:6837","nonce":3103107210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4282":{"gid":4282,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6838/3621458364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3621458364},{"type":"v1","addr":"172.21.15.113:6838","nonce":3621458364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.47:6835/1117134154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1117134154},{"type":"v1","addr":"172.21.15.47:6835","nonce":1117134154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4238,4241,4306,4279,4282]},"id":1}]} 2024-04-24T22:44:04.172 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2024-04-24T22:44:04.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 -- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0de406be30 msgr2=0x7f0de406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:04.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0de406be30 0x7f0de406e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0e041ede50 tx=0x7f0df0009040 comp rx=0 tx=0).stop 2024-04-24T22:44:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 -- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0406e6f0 msgr2=0x7f0e04152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0406e6f0 0x7f0e04152520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0e0415b5f0 tx=0x7f0df80046c0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 -- 172.21.15.47:0/4086248598 shutdown_connections 2024-04-24T22:44:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f0de406be30 0x7f0de406e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e041514f0 0x7f0e040c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e040c6a20 0x7f0e041e0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 --2- 172.21.15.47:0/4086248598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0406e6f0 0x7f0e04152520 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 -- 172.21.15.47:0/4086248598 >> 172.21.15.47:0/4086248598 conn(0x7f0e040cb900 msgr2=0x7f0e040cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 -- 172.21.15.47:0/4086248598 shutdown_connections 2024-04-24T22:44:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.171+0000 7f0e0a800640 1 -- 172.21.15.47:0/4086248598 wait complete. 2024-04-24T22:44:04.195 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-24T22:44:04.195 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-24T22:44:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.394+0000 7fa1da000640 1 Processor -- start 2024-04-24T22:44:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.394+0000 7fa1da000640 1 -- start start 2024-04-24T22:44:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.396+0000 7fa1da000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d415a680 0x7fa1d40c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.396+0000 7fa1da000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa1d40c8ad0 0x7fa1d40c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.396+0000 7fa1da000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d415ab10 con 0x7fa1d4154960 2024-04-24T22:44:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.396+0000 7fa1da000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d415adf0 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.396+0000 7fa1da000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d415b0d0 con 0x7fa1d415a680 2024-04-24T22:44:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.396+0000 7fa1d3400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa1d4154960 0x7fa1d415e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33266/0 (socket says 172.21.15.47:33266) 2024-04-24T22:44:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.396+0000 7fa1d3400640 1 -- 172.21.15.47:0/4111791845 learned_addr learned my addr 172.21.15.47:0/4111791845 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.396+0000 7fa1d3e00640 1 --2- 172.21.15.47:0/4111791845 >> v2:172.21.15.47:3300/0 conn(0x7fa1d40c8ad0 0x7fa1d40c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:04.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.396+0000 7fa1d2a00640 1 --2- 172.21.15.47:0/4111791845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d415a680 0x7fa1d40c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:04.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.397+0000 7fa1d2000640 1 -- 172.21.15.47:0/4111791845 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 183523211 0 0) 0x7fa1d415ab10 con 0x7fa1d4154960 2024-04-24T22:44:04.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.397+0000 7fa1d2a00640 1 -- 172.21.15.47:0/4111791845 >> v2:172.21.15.47:3300/0 conn(0x7fa1d40c8ad0 msgr2=0x7fa1d40c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:04.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.397+0000 7fa1d2a00640 1 --2- 172.21.15.47:0/4111791845 >> v2:172.21.15.47:3300/0 conn(0x7fa1d40c8ad0 0x7fa1d40c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.397+0000 7fa1d2a00640 1 -- 172.21.15.47:0/4111791845 >> v1:172.21.15.47:6789/0 conn(0x7fa1d4154960 legacy=0x7fa1d415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:04.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.397+0000 7fa1d2a00640 1 -- 172.21.15.47:0/4111791845 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d415b3b0 con 0x7fa1d415a680 2024-04-24T22:44:04.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.398+0000 7fa1d2a00640 1 --2- 172.21.15.47:0/4111791845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d415a680 0x7fa1d40c8590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00097a0 tx=0x7fa1c002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6bf9d0980c29d9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:04.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.398+0000 7fa1d2000640 1 -- 172.21.15.47:0/4111791845 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0037070 con 0x7fa1d415a680 2024-04-24T22:44:04.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.398+0000 7fa1d2000640 1 -- 172.21.15.47:0/4111791845 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c0032440 con 0x7fa1d415a680 2024-04-24T22:44:04.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.398+0000 7fa1d2000640 1 -- 172.21.15.47:0/4111791845 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0032810 con 0x7fa1d415a680 2024-04-24T22:44:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.399+0000 7fa1da000640 1 -- 172.21.15.47:0/4111791845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d415a680 msgr2=0x7fa1d40c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.399+0000 7fa1da000640 1 --2- 172.21.15.47:0/4111791845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d415a680 0x7fa1d40c8590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00097a0 tx=0x7fa1c002a180 comp rx=0 tx=0).stop 2024-04-24T22:44:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.399+0000 7fa1da000640 1 -- 172.21.15.47:0/4111791845 shutdown_connections 2024-04-24T22:44:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.399+0000 7fa1da000640 1 --2- 172.21.15.47:0/4111791845 >> v2:172.21.15.47:3300/0 conn(0x7fa1d40c8ad0 0x7fa1d40c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.399+0000 7fa1da000640 1 --2- 172.21.15.47:0/4111791845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d415a680 0x7fa1d40c8590 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.399+0000 7fa1da000640 1 -- 172.21.15.47:0/4111791845 >> 172.21.15.47:0/4111791845 conn(0x7fa1d406e580 msgr2=0x7fa1d406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.399+0000 7fa1da000640 1 -- 172.21.15.47:0/4111791845 shutdown_connections 2024-04-24T22:44:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.399+0000 7fa1da000640 1 -- 172.21.15.47:0/4111791845 wait complete. 2024-04-24T22:44:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.401+0000 7fa1da000640 1 Processor -- start 2024-04-24T22:44:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.402+0000 7fa1da000640 1 -- start start 2024-04-24T22:44:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1da000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d40c8ad0 0x7fa1d41f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1da000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d4154960 0x7fa1d41e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1da000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d41e5a30 0x7fa1d4152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1da000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d415ea00 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1da000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d4154d40 con 0x7fa1d41e5a30 2024-04-24T22:44:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1da000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d415eb80 con 0x7fa1d4154960 2024-04-24T22:44:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d40c8ad0 0x7fa1d41f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d40c8ad0 0x7fa1d41f0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51218/0 (socket says 172.21.15.47:51218) 2024-04-24T22:44:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1d3400640 1 -- 172.21.15.47:0/3282378461 learned_addr learned my addr 172.21.15.47:0/3282378461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1d3e00640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d41e5a30 0x7fa1d4152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.403+0000 7fa1d2a00640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d4154960 0x7fa1d41e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1d3400640 1 -- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d4154960 msgr2=0x7fa1d41e54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1d3400640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d4154960 0x7fa1d41e54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1d3400640 1 -- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d41e5a30 msgr2=0x7fa1d4152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1d3400640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d41e5a30 0x7fa1d4152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1d3400640 1 -- 172.21.15.47:0/3282378461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d41e5ec0 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1d3400640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d40c8ad0 0x7fa1d41f0aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400c500 tx=0x7fa1c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1cb400640 1 -- 172.21.15.47:0/3282378461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4018070 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1cb400640 1 -- 172.21.15.47:0/3282378461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c4007580 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1cb400640 1 -- 172.21.15.47:0/3282378461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4004a20 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1da000640 1 -- 172.21.15.47:0/3282378461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d41e78f0 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.404+0000 7fa1da000640 1 -- 172.21.15.47:0/3282378461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d4151f30 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.406+0000 7fa1da000640 1 -- 172.21.15.47:0/3282378461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d4073b90 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.407+0000 7fa1cb400640 1 -- 172.21.15.47:0/3282378461 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1c4004bc0 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.408+0000 7fa1cb400640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa1b406be30 0x7fa1b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:04.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.408+0000 7fa1cb400640 1 -- 172.21.15.47:0/3282378461 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa1c408a180 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.408+0000 7fa1d2a00640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa1b406be30 0x7fa1b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.409+0000 7fa1d2a00640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa1b406be30 0x7fa1b406e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0034000 tx=0x7fa1c002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:04.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.415+0000 7fa1cb400640 1 -- 172.21.15.47:0/3282378461 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1c40583c0 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.693+0000 7fa1da000640 1 -- 172.21.15.47:0/3282378461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa1d40c76d0 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.694+0000 7fa1cb400640 1 -- 172.21.15.47:0/3282378461 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fa1c405c6c0 con 0x7fa1d40c8ad0 2024-04-24T22:44:04.697 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","modified":"2024-04-24T22:38:41.976658Z","created":"2024-04-24T22:38:41.976658Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3301","nonce":0},{"type":"v1","addr":"172.21.15.113:6790","nonce":0}]},"addr":"172.21.15.113:6790/0","public_addr":"172.21.15.113:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-24T22:44:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.696+0000 7fa1da000640 1 -- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa1b406be30 msgr2=0x7fa1b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.696+0000 7fa1da000640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa1b406be30 0x7fa1b406e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0034000 tx=0x7fa1c002e000 comp rx=0 tx=0).stop 2024-04-24T22:44:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.696+0000 7fa1da000640 1 -- 172.21.15.47:0/3282378461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d40c8ad0 msgr2=0x7fa1d41f0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.696+0000 7fa1da000640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d40c8ad0 0x7fa1d41f0aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400c500 tx=0x7fa1c400c9c0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.697+0000 7fa1da000640 1 -- 172.21.15.47:0/3282378461 shutdown_connections 2024-04-24T22:44:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.697+0000 7fa1da000640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa1b406be30 0x7fa1b406e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.697+0000 7fa1da000640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d41e5a30 0x7fa1d4152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.697+0000 7fa1da000640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d4154960 0x7fa1d41e54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.697+0000 7fa1da000640 1 --2- 172.21.15.47:0/3282378461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1d40c8ad0 0x7fa1d41f0aa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.697+0000 7fa1da000640 1 -- 172.21.15.47:0/3282378461 >> 172.21.15.47:0/3282378461 conn(0x7fa1d406e580 msgr2=0x7fa1d4072120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.697+0000 7fa1da000640 1 -- 172.21.15.47:0/3282378461 shutdown_connections 2024-04-24T22:44:04.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:04.697+0000 7fa1da000640 1 -- 172.21.15.47:0/3282378461 wait complete. 2024-04-24T22:44:04.719 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-24T22:44:04.720 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-24T22:44:04.729 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-24T22:44:04.729 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:44:04.729 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:44:04.789 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-24T22:44:04.789 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T22:44:04.844 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:44:05.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.061+0000 7fa514200640 1 Processor -- start 2024-04-24T22:44:05.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.062+0000 7fa514200640 1 -- start start 2024-04-24T22:44:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.063+0000 7fa514200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c152960 0x7fa50c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.063+0000 7fa514200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa50c072b80 0x7fa50c073010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.063+0000 7fa514200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa50c0c9400 con 0x7fa50c0c8fa0 2024-04-24T22:44:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.063+0000 7fa514200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa50c158f20 con 0x7fa50c072b80 2024-04-24T22:44:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.063+0000 7fa514200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa50c1593b0 con 0x7fa50c152960 2024-04-24T22:44:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.064+0000 7fa512200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa50c072b80 0x7fa50c073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.064+0000 7fa512200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa50c072b80 0x7fa50c073010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51226/0 (socket says 172.21.15.47:51226) 2024-04-24T22:44:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.064+0000 7fa512200640 1 -- 172.21.15.47:0/3062294100 learned_addr learned my addr 172.21.15.47:0/3062294100 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.064+0000 7fa511800640 1 --2- 172.21.15.47:0/3062294100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c152960 0x7fa50c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.064+0000 7fa512200640 1 -- 172.21.15.47:0/3062294100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c152960 msgr2=0x7fa50c0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.064+0000 7fa512200640 1 --2- 172.21.15.47:0/3062294100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c152960 0x7fa50c0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.064+0000 7fa50be00640 1 -- 172.21.15.47:0/3062294100 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336132955 0 0) 0x7fa50c0c9400 con 0x7fa50c0c8fa0 2024-04-24T22:44:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.064+0000 7fa512200640 1 -- 172.21.15.47:0/3062294100 >> v1:172.21.15.113:6789/0 conn(0x7fa50c0c8fa0 legacy=0x7fa50c15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.064+0000 7fa512200640 1 -- 172.21.15.47:0/3062294100 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c0c7e50 con 0x7fa50c072b80 2024-04-24T22:44:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.064+0000 7fa511800640 1 --2- 172.21.15.47:0/3062294100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c152960 0x7fa50c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:44:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.066+0000 7fa512200640 1 --2- 172.21.15.47:0/3062294100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa50c072b80 0x7fa50c073010 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa504009e50 tx=0x7fa50402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ae85c97ee91e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.066+0000 7fa50be00640 1 -- 172.21.15.47:0/3062294100 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504036070 con 0x7fa50c072b80 2024-04-24T22:44:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.066+0000 7fa50be00640 1 -- 172.21.15.47:0/3062294100 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa504031600 con 0x7fa50c072b80 2024-04-24T22:44:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.066+0000 7fa50be00640 1 -- 172.21.15.47:0/3062294100 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5040319d0 con 0x7fa50c072b80 2024-04-24T22:44:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.067+0000 7fa514200640 1 -- 172.21.15.47:0/3062294100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa50c072b80 msgr2=0x7fa50c073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.067+0000 7fa514200640 1 --2- 172.21.15.47:0/3062294100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa50c072b80 0x7fa50c073010 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa504009e50 tx=0x7fa50402a7a0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.067+0000 7fa514200640 1 -- 172.21.15.47:0/3062294100 shutdown_connections 2024-04-24T22:44:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.067+0000 7fa514200640 1 --2- 172.21.15.47:0/3062294100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa50c072b80 0x7fa50c073010 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.067+0000 7fa514200640 1 --2- 172.21.15.47:0/3062294100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c152960 0x7fa50c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.067+0000 7fa514200640 1 -- 172.21.15.47:0/3062294100 >> 172.21.15.47:0/3062294100 conn(0x7fa50c06e580 msgr2=0x7fa50c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.067+0000 7fa514200640 1 -- 172.21.15.47:0/3062294100 shutdown_connections 2024-04-24T22:44:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.067+0000 7fa514200640 1 -- 172.21.15.47:0/3062294100 wait complete. 2024-04-24T22:44:05.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.068+0000 7fa514200640 1 Processor -- start 2024-04-24T22:44:05.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.069+0000 7fa514200640 1 -- start start 2024-04-24T22:44:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.069+0000 7fa514200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c072b80 0x7fa50c1f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa514200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa50c0c8fa0 0x7fa50c1f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa514200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa50c152960 0x7fa50c070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa514200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1588d0 con 0x7fa50c152960 2024-04-24T22:44:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa514200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0c8130 con 0x7fa50c0c8fa0 2024-04-24T22:44:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa514200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa50c158bb0 con 0x7fa50c072b80 2024-04-24T22:44:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa512200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa50c152960 0x7fa50c070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa511800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c072b80 0x7fa50c1f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa510e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa50c0c8fa0 0x7fa50c1f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa511800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c072b80 0x7fa50c1f0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:50306/0 (socket says 172.21.15.47:50306) 2024-04-24T22:44:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa511800640 1 -- 172.21.15.47:0/4288018354 learned_addr learned my addr 172.21.15.47:0/4288018354 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa512200640 1 -- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c072b80 msgr2=0x7fa50c1f0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa512200640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c072b80 0x7fa50c1f0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa512200640 1 -- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa50c0c8fa0 msgr2=0x7fa50c1f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa512200640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa50c0c8fa0 0x7fa50c1f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.070+0000 7fa512200640 1 -- 172.21.15.47:0/4288018354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c1f1b00 con 0x7fa50c152960 2024-04-24T22:44:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.071+0000 7fa512200640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa50c152960 0x7fa50c070950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa504030000 tx=0x7fa504004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.071+0000 7fa509600640 1 -- 172.21.15.47:0/4288018354 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504036050 con 0x7fa50c152960 2024-04-24T22:44:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.071+0000 7fa509600640 1 -- 172.21.15.47:0/4288018354 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa504042070 con 0x7fa50c152960 2024-04-24T22:44:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.071+0000 7fa509600640 1 -- 172.21.15.47:0/4288018354 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504009510 con 0x7fa50c152960 2024-04-24T22:44:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.071+0000 7fa514200640 1 -- 172.21.15.47:0/4288018354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c0cb300 con 0x7fa50c152960 2024-04-24T22:44:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.071+0000 7fa514200640 1 -- 172.21.15.47:0/4288018354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c0cb880 con 0x7fa50c152960 2024-04-24T22:44:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.072+0000 7fa514200640 1 -- 172.21.15.47:0/4288018354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa50c073b90 con 0x7fa50c152960 2024-04-24T22:44:05.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.073+0000 7fa509600640 1 -- 172.21.15.47:0/4288018354 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa504007bf0 con 0x7fa50c152960 2024-04-24T22:44:05.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.074+0000 7fa509600640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4ec06be30 0x7fa4ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.074+0000 7fa509600640 1 -- 172.21.15.47:0/4288018354 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa5040abc80 con 0x7fa50c152960 2024-04-24T22:44:05.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.075+0000 7fa511800640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4ec06be30 0x7fa4ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.075+0000 7fa511800640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4ec06be30 0x7fa4ec06e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc006d40 tx=0x7fa4fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:05.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.079+0000 7fa509600640 1 -- 172.21.15.47:0/4288018354 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa504079e30 con 0x7fa50c152960 2024-04-24T22:44:05.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.287+0000 7fa514200640 1 -- 172.21.15.47:0/4288018354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa50c06c200 con 0x7fa50c152960 2024-04-24T22:44:05.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.288+0000 7fa509600640 1 -- 172.21.15.47:0/4288018354 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16843 (secure 0 0 0) 0x7fa50407e130 con 0x7fa50c152960 2024-04-24T22:44:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:05.292 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:40:23.897411+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:44:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 -- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4ec06be30 msgr2=0x7fa4ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4ec06be30 0x7fa4ec06e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc006d40 tx=0x7fa4fc008040 comp rx=0 tx=0).stop 2024-04-24T22:44:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 -- 172.21.15.47:0/4288018354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa50c152960 msgr2=0x7fa50c070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa50c152960 0x7fa50c070950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa504030000 tx=0x7fa504004a40 comp rx=0 tx=0).stop 2024-04-24T22:44:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 -- 172.21.15.47:0/4288018354 shutdown_connections 2024-04-24T22:44:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa4ec06be30 0x7fa4ec06e2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa50c152960 0x7fa50c070950 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa50c0c8fa0 0x7fa50c1f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 --2- 172.21.15.47:0/4288018354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa50c072b80 0x7fa50c1f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 -- 172.21.15.47:0/4288018354 >> 172.21.15.47:0/4288018354 conn(0x7fa50c06e580 msgr2=0x7fa50c072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 -- 172.21.15.47:0/4288018354 shutdown_connections 2024-04-24T22:44:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.291+0000 7fa514200640 1 -- 172.21.15.47:0/4288018354 wait complete. 2024-04-24T22:44:05.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T22:39:30.711538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T22:39:34.803185+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T22:39:36.184123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T22:44:05.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T22:44:05.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.525+0000 7f2339400640 1 Processor -- start 2024-04-24T22:44:05.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.525+0000 7f2339400640 1 -- start start 2024-04-24T22:44:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.527+0000 7f2339400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334154560 0x7f23340cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.527+0000 7f2339400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2334073280 0x7f23340cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.527+0000 7f2339400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f233415b2c0 con 0x7f23340cc660 2024-04-24T22:44:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.527+0000 7f2339400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f233415da90 con 0x7f2334073280 2024-04-24T22:44:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.527+0000 7f2339400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f233415dd70 con 0x7f2334154560 2024-04-24T22:44:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.527+0000 7f2333e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334154560 0x7f23340cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.527+0000 7f2333400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2334073280 0x7f23340cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.527+0000 7f2338a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f23340cc660 0x7f23340ccaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:39510/0 (socket says 172.21.15.47:39510) 2024-04-24T22:44:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.527+0000 7f2338a00640 1 -- 172.21.15.47:0/2288388143 learned_addr learned my addr 172.21.15.47:0/2288388143 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.527+0000 7f2333e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334154560 0x7f23340cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:50318/0 (socket says 172.21.15.47:50318) 2024-04-24T22:44:05.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.528+0000 7f2333e00640 1 -- 172.21.15.47:0/2288388143 >> v2:172.21.15.47:3300/0 conn(0x7f2334073280 msgr2=0x7f23340cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:05.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.528+0000 7f2333e00640 1 --2- 172.21.15.47:0/2288388143 >> v2:172.21.15.47:3300/0 conn(0x7f2334073280 0x7f23340cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.528+0000 7f2333e00640 1 -- 172.21.15.47:0/2288388143 >> v1:172.21.15.113:6789/0 conn(0x7f23340cc660 legacy=0x7f23340ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.528+0000 7f2333e00640 1 -- 172.21.15.47:0/2288388143 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233415e050 con 0x7f2334154560 2024-04-24T22:44:05.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.528+0000 7f2332a00640 1 -- 172.21.15.47:0/2288388143 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 243550094 0 0) 0x7f233415b2c0 con 0x7f23340cc660 2024-04-24T22:44:05.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.529+0000 7f2333e00640 1 --2- 172.21.15.47:0/2288388143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334154560 0x7f23340cb430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23240097a0 tx=0x7f232402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9de724d0f02526 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:05.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.529+0000 7f2332a00640 1 -- 172.21.15.47:0/2288388143 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324037070 con 0x7f2334154560 2024-04-24T22:44:05.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.529+0000 7f2332a00640 1 -- 172.21.15.47:0/2288388143 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2324032440 con 0x7f2334154560 2024-04-24T22:44:05.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.529+0000 7f2332a00640 1 -- 172.21.15.47:0/2288388143 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324032810 con 0x7f2334154560 2024-04-24T22:44:05.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.530+0000 7f2339400640 1 -- 172.21.15.47:0/2288388143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334154560 msgr2=0x7f23340cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.530+0000 7f2339400640 1 --2- 172.21.15.47:0/2288388143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334154560 0x7f23340cb430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23240097a0 tx=0x7f232402a180 comp rx=0 tx=0).stop 2024-04-24T22:44:05.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.530+0000 7f2339400640 1 -- 172.21.15.47:0/2288388143 shutdown_connections 2024-04-24T22:44:05.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.530+0000 7f2339400640 1 --2- 172.21.15.47:0/2288388143 >> v2:172.21.15.47:3300/0 conn(0x7f2334073280 0x7f23340cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.530+0000 7f2339400640 1 --2- 172.21.15.47:0/2288388143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334154560 0x7f23340cb430 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.530+0000 7f2339400640 1 -- 172.21.15.47:0/2288388143 >> 172.21.15.47:0/2288388143 conn(0x7f23340672c0 msgr2=0x7f2334072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:05.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.530+0000 7f2339400640 1 -- 172.21.15.47:0/2288388143 shutdown_connections 2024-04-24T22:44:05.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.530+0000 7f2339400640 1 -- 172.21.15.47:0/2288388143 wait complete. 2024-04-24T22:44:05.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.532+0000 7f2339400640 1 Processor -- start 2024-04-24T22:44:05.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.532+0000 7f2339400640 1 -- start start 2024-04-24T22:44:05.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.533+0000 7f2339400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334073280 0x7f233415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.533+0000 7f2339400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23340cc660 0x7f2334143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.533+0000 7f2339400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2334154560 0x7f2334143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.533+0000 7f2339400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f23340cd200 con 0x7f23340cc660 2024-04-24T22:44:05.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.533+0000 7f2339400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23340caad0 con 0x7f2334154560 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.533+0000 7f2339400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f23340cd380 con 0x7f2334073280 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.533+0000 7f2333400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23340cc660 0x7f2334143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.533+0000 7f2333400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23340cc660 0x7f2334143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51246/0 (socket says 172.21.15.47:51246) 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.533+0000 7f2333400640 1 -- 172.21.15.47:0/1738541596 learned_addr learned my addr 172.21.15.47:0/1738541596 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.534+0000 7f2333e00640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334073280 0x7f233415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.534+0000 7f2338a00640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2334154560 0x7f2334143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.534+0000 7f2333400640 1 -- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334073280 msgr2=0x7f233415b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.534+0000 7f2333400640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334073280 0x7f233415b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.534+0000 7f2333400640 1 -- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2334154560 msgr2=0x7f2334143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.534+0000 7f2333400640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2334154560 0x7f2334143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.534+0000 7f2333400640 1 -- 172.21.15.47:0/1738541596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2334159480 con 0x7f23340cc660 2024-04-24T22:44:05.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.534+0000 7f2333400640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23340cc660 0x7f2334143650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f231c00efc0 tx=0x7f231c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:05.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.534+0000 7f232be00640 1 -- 172.21.15.47:0/1738541596 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c018020 con 0x7f23340cc660 2024-04-24T22:44:05.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.535+0000 7f232be00640 1 -- 172.21.15.47:0/1738541596 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c014480 con 0x7f23340cc660 2024-04-24T22:44:05.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.535+0000 7f232be00640 1 -- 172.21.15.47:0/1738541596 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c014850 con 0x7f23340cc660 2024-04-24T22:44:05.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.535+0000 7f2339400640 1 -- 172.21.15.47:0/1738541596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23341446b0 con 0x7f23340cc660 2024-04-24T22:44:05.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.535+0000 7f2339400640 1 -- 172.21.15.47:0/1738541596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2334159a00 con 0x7f23340cc660 2024-04-24T22:44:05.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.536+0000 7f2339400640 1 -- 172.21.15.47:0/1738541596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233415a2a0 con 0x7f23340cc660 2024-04-24T22:44:05.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.537+0000 7f232be00640 1 -- 172.21.15.47:0/1738541596 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f231c005d10 con 0x7f23340cc660 2024-04-24T22:44:05.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.538+0000 7f232be00640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f231006be80 0x7f231006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.538+0000 7f232be00640 1 -- 172.21.15.47:0/1738541596 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f231c08bb10 con 0x7f23340cc660 2024-04-24T22:44:05.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.538+0000 7f2333e00640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f231006be80 0x7f231006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.539+0000 7f2333e00640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f231006be80 0x7f231006e340 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2324034000 tx=0x7f232402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:05.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.543+0000 7f232be00640 1 -- 172.21.15.47:0/1738541596 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f231c022030 con 0x7f23340cc660 2024-04-24T22:44:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.744+0000 7f2339400640 1 -- 172.21.15.47:0/1738541596 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2334147660 con 0x7f23340cc660 2024-04-24T22:44:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.745+0000 7f232be00640 1 -- 172.21.15.47:0/1738541596 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f231c010040 con 0x7f23340cc660 2024-04-24T22:44:05.748 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-24T22:44:05.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.747+0000 7f2339400640 1 -- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f231006be80 msgr2=0x7f231006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.747+0000 7f2339400640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f231006be80 0x7f231006e340 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2324034000 tx=0x7f232402e000 comp rx=0 tx=0).stop 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.748+0000 7f2339400640 1 -- 172.21.15.47:0/1738541596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23340cc660 msgr2=0x7f2334143650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.748+0000 7f2339400640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23340cc660 0x7f2334143650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f231c00efc0 tx=0x7f231c00c490 comp rx=0 tx=0).stop 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.748+0000 7f2339400640 1 -- 172.21.15.47:0/1738541596 shutdown_connections 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.748+0000 7f2339400640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f231006be80 0x7f231006e340 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.748+0000 7f2339400640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2334154560 0x7f2334143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.748+0000 7f2339400640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23340cc660 0x7f2334143650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.748+0000 7f2339400640 1 --2- 172.21.15.47:0/1738541596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2334073280 0x7f233415b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.748+0000 7f2339400640 1 -- 172.21.15.47:0/1738541596 >> 172.21.15.47:0/1738541596 conn(0x7f23340672c0 msgr2=0x7f233414a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.748+0000 7f2339400640 1 -- 172.21.15.47:0/1738541596 shutdown_connections 2024-04-24T22:44:05.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.748+0000 7f2339400640 1 -- 172.21.15.47:0/1738541596 wait complete. 2024-04-24T22:44:05.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T22:44:05.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.989+0000 7feb9d600640 1 Processor -- start 2024-04-24T22:44:05.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.989+0000 7feb9d600640 1 -- start start 2024-04-24T22:44:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.990+0000 7feb9d600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb980cc840 0x7feb980ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.990+0000 7feb9d600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb980cbde0 0x7feb980ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.990+0000 7feb9d600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb9815b240 con 0x7feb98073380 2024-04-24T22:44:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.990+0000 7feb9d600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7feb9815daa0 con 0x7feb980cbde0 2024-04-24T22:44:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.990+0000 7feb9d600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7feb9815dcf0 con 0x7feb980cc840 2024-04-24T22:44:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.991+0000 7feb96a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb980cc840 0x7feb980ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.991+0000 7feb96a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb980cc840 0x7feb980ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60112/0 (socket says 172.21.15.47:60112) 2024-04-24T22:44:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.991+0000 7feb96a00640 1 -- 172.21.15.47:0/3965690762 learned_addr learned my addr 172.21.15.47:0/3965690762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.991+0000 7feb97400640 1 --2- 172.21.15.47:0/3965690762 >> v2:172.21.15.47:3300/0 conn(0x7feb980cbde0 0x7feb980ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:05.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.991+0000 7feb96a00640 1 -- 172.21.15.47:0/3965690762 >> v2:172.21.15.47:3300/0 conn(0x7feb980cbde0 msgr2=0x7feb980ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:05.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.991+0000 7feb96a00640 1 --2- 172.21.15.47:0/3965690762 >> v2:172.21.15.47:3300/0 conn(0x7feb980cbde0 0x7feb980ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.991+0000 7feb96a00640 1 -- 172.21.15.47:0/3965690762 >> v1:172.21.15.47:6789/0 conn(0x7feb98073380 legacy=0x7feb980cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:05.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.991+0000 7feb96a00640 1 -- 172.21.15.47:0/3965690762 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9815dfd0 con 0x7feb980cc840 2024-04-24T22:44:05.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.991+0000 7feb97400640 1 --2- 172.21.15.47:0/3965690762 >> v2:172.21.15.47:3300/0 conn(0x7feb980cbde0 0x7feb980ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:44:05.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.993+0000 7feb96a00640 1 --2- 172.21.15.47:0/3965690762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb980cc840 0x7feb980ca7d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feb98154d70 tx=0x7feb8802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4013a1e59186852a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:05.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.993+0000 7feb95600640 1 -- 172.21.15.47:0/3965690762 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88037070 con 0x7feb980cc840 2024-04-24T22:44:05.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.993+0000 7feb95600640 1 -- 172.21.15.47:0/3965690762 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb88032440 con 0x7feb980cc840 2024-04-24T22:44:05.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.993+0000 7feb95600640 1 -- 172.21.15.47:0/3965690762 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88032810 con 0x7feb980cc840 2024-04-24T22:44:05.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.994+0000 7feb9d600640 1 -- 172.21.15.47:0/3965690762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb980cc840 msgr2=0x7feb980ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:05.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.994+0000 7feb9d600640 1 --2- 172.21.15.47:0/3965690762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb980cc840 0x7feb980ca7d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feb98154d70 tx=0x7feb8802a270 comp rx=0 tx=0).stop 2024-04-24T22:44:05.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.994+0000 7feb9d600640 1 -- 172.21.15.47:0/3965690762 shutdown_connections 2024-04-24T22:44:05.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.994+0000 7feb9d600640 1 --2- 172.21.15.47:0/3965690762 >> v2:172.21.15.47:3300/0 conn(0x7feb980cbde0 0x7feb980ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.994+0000 7feb9d600640 1 --2- 172.21.15.47:0/3965690762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb980cc840 0x7feb980ca7d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:05.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.994+0000 7feb9d600640 1 -- 172.21.15.47:0/3965690762 >> 172.21.15.47:0/3965690762 conn(0x7feb9806c940 msgr2=0x7feb98072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:05.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.994+0000 7feb9d600640 1 -- 172.21.15.47:0/3965690762 shutdown_connections 2024-04-24T22:44:05.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.994+0000 7feb9d600640 1 -- 172.21.15.47:0/3965690762 wait complete. 2024-04-24T22:44:05.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.996+0000 7feb9d600640 1 Processor -- start 2024-04-24T22:44:05.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.996+0000 7feb9d600640 1 -- start start 2024-04-24T22:44:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb9d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb98073380 0x7feb980cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb9d600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb980cbde0 0x7feb981e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb9d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb980cc840 0x7feb9814a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb9d600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb9815e870 con 0x7feb980cc840 2024-04-24T22:44:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb9d600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feb9815e590 con 0x7feb98073380 2024-04-24T22:44:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb9d600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7feb9815af30 con 0x7feb980cbde0 2024-04-24T22:44:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb96a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb98073380 0x7feb980cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb96a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb98073380 0x7feb980cedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60128/0 (socket says 172.21.15.47:60128) 2024-04-24T22:44:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb96a00640 1 -- 172.21.15.47:0/2669234010 learned_addr learned my addr 172.21.15.47:0/2669234010 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb96000640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb980cbde0 0x7feb981e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb97400640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb980cc840 0x7feb9814a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb96a00640 1 -- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb980cbde0 msgr2=0x7feb981e5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb96a00640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb980cbde0 0x7feb981e5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb96a00640 1 -- 172.21.15.47:0/2669234010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb980cc840 msgr2=0x7feb9814a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb96a00640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb980cc840 0x7feb9814a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.998+0000 7feb96a00640 1 -- 172.21.15.47:0/2669234010 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb980cdeb0 con 0x7feb98073380 2024-04-24T22:44:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.999+0000 7feb96a00640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb98073380 0x7feb980cedd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feb8802a2b0 tx=0x7feb880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.999+0000 7feb8ea00640 1 -- 172.21.15.47:0/2669234010 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88041020 con 0x7feb98073380 2024-04-24T22:44:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.999+0000 7feb8ea00640 1 -- 172.21.15.47:0/2669234010 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb880306c0 con 0x7feb98073380 2024-04-24T22:44:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.999+0000 7feb8ea00640 1 -- 172.21.15.47:0/2669234010 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88037040 con 0x7feb98073380 2024-04-24T22:44:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.999+0000 7feb9d600640 1 -- 172.21.15.47:0/2669234010 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb980ce190 con 0x7feb98073380 2024-04-24T22:44:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:05.999+0000 7feb9d600640 1 -- 172.21.15.47:0/2669234010 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb980ce750 con 0x7feb98073380 2024-04-24T22:44:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.000+0000 7feb9d600640 1 -- 172.21.15.47:0/2669234010 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb98073b90 con 0x7feb98073380 2024-04-24T22:44:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.001+0000 7feb8ea00640 1 -- 172.21.15.47:0/2669234010 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb88044020 con 0x7feb98073380 2024-04-24T22:44:06.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.002+0000 7feb8ea00640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feb7406be30 0x7feb7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.002+0000 7feb8ea00640 1 -- 172.21.15.47:0/2669234010 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7feb880ab740 con 0x7feb98073380 2024-04-24T22:44:06.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.002+0000 7feb96000640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feb7406be30 0x7feb7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.003+0000 7feb96000640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feb7406be30 0x7feb7406e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7feb9815da00 tx=0x7feb80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:06.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.007+0000 7feb8ea00640 1 -- 172.21.15.47:0/2669234010 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feb8804b030 con 0x7feb98073380 2024-04-24T22:44:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.214+0000 7feb9d600640 1 -- 172.21.15.47:0/2669234010 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feb980cadb0 con 0x7feb98073380 2024-04-24T22:44:06.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.214+0000 7feb8ea00640 1 -- 172.21.15.47:0/2669234010 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7feb88079fd0 con 0x7feb98073380 2024-04-24T22:44:06.217 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-24T22:44:06.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.217+0000 7feb9d600640 1 -- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feb7406be30 msgr2=0x7feb7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.217+0000 7feb9d600640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feb7406be30 0x7feb7406e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7feb9815da00 tx=0x7feb80009040 comp rx=0 tx=0).stop 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.218+0000 7feb9d600640 1 -- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb98073380 msgr2=0x7feb980cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.218+0000 7feb9d600640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb98073380 0x7feb980cedd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feb8802a2b0 tx=0x7feb880046c0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.218+0000 7feb9d600640 1 -- 172.21.15.47:0/2669234010 shutdown_connections 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.218+0000 7feb9d600640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7feb7406be30 0x7feb7406e2f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.218+0000 7feb9d600640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb980cc840 0x7feb9814a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.218+0000 7feb9d600640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb980cbde0 0x7feb981e5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.218+0000 7feb9d600640 1 --2- 172.21.15.47:0/2669234010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb98073380 0x7feb980cedd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.218+0000 7feb9d600640 1 -- 172.21.15.47:0/2669234010 >> 172.21.15.47:0/2669234010 conn(0x7feb9806c940 msgr2=0x7feb981e38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.218+0000 7feb9d600640 1 -- 172.21.15.47:0/2669234010 shutdown_connections 2024-04-24T22:44:06.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.218+0000 7feb9d600640 1 -- 172.21.15.47:0/2669234010 wait complete. 2024-04-24T22:44:06.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T22:44:06.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.456+0000 7f4235e00640 1 Processor -- start 2024-04-24T22:44:06.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.456+0000 7f4235e00640 1 -- start start 2024-04-24T22:44:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f4235e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4230072c60 0x7f42300c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f4235e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4230073040 0x7f42301411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f4235e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f423006bb90 0x7f4230141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f4235e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4230158bf0 con 0x7f4230072c60 2024-04-24T22:44:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f4235e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f423015c3d0 con 0x7f423006bb90 2024-04-24T22:44:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f4235e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f423015c6b0 con 0x7f4230073040 2024-04-24T22:44:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f4234a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4230072c60 0x7f42300c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f4234a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4230072c60 0x7f42300c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35532/0 (socket says 172.21.15.47:35532) 2024-04-24T22:44:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f4234a00640 1 -- 172.21.15.47:0/1886485728 learned_addr learned my addr 172.21.15.47:0/1886485728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f422fe00640 1 --2- 172.21.15.47:0/1886485728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4230073040 0x7f42301411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.458+0000 7f4235400640 1 --2- 172.21.15.47:0/1886485728 >> v2:172.21.15.113:3300/0 conn(0x7f423006bb90 0x7f4230141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.459+0000 7f4234a00640 1 -- 172.21.15.47:0/1886485728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4230073040 msgr2=0x7f42301411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.459+0000 7f4234a00640 1 --2- 172.21.15.47:0/1886485728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4230073040 0x7f42301411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.459+0000 7f4234a00640 1 -- 172.21.15.47:0/1886485728 >> v2:172.21.15.113:3300/0 conn(0x7f423006bb90 msgr2=0x7f4230141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.459+0000 7f4234a00640 1 --2- 172.21.15.47:0/1886485728 >> v2:172.21.15.113:3300/0 conn(0x7f423006bb90 0x7f4230141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.459+0000 7f4234a00640 1 -- 172.21.15.47:0/1886485728 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42301585a0 con 0x7f4230072c60 2024-04-24T22:44:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.460+0000 7f4234a00640 1 --2- 172.21.15.47:0/1886485728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4230072c60 0x7f42300c2d60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f42300c97b0 tx=0x7f422402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6376345e3d300de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.460+0000 7f422f400640 1 -- 172.21.15.47:0/1886485728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224037070 con 0x7f4230072c60 2024-04-24T22:44:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.460+0000 7f422f400640 1 -- 172.21.15.47:0/1886485728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4224032440 con 0x7f4230072c60 2024-04-24T22:44:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.460+0000 7f422f400640 1 -- 172.21.15.47:0/1886485728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224032810 con 0x7f4230072c60 2024-04-24T22:44:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.461+0000 7f4235e00640 1 -- 172.21.15.47:0/1886485728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4230072c60 msgr2=0x7f42300c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.461+0000 7f4235e00640 1 --2- 172.21.15.47:0/1886485728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4230072c60 0x7f42300c2d60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f42300c97b0 tx=0x7f422402a270 comp rx=0 tx=0).stop 2024-04-24T22:44:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.461+0000 7f4235e00640 1 -- 172.21.15.47:0/1886485728 shutdown_connections 2024-04-24T22:44:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.461+0000 7f4235e00640 1 --2- 172.21.15.47:0/1886485728 >> v2:172.21.15.113:3300/0 conn(0x7f423006bb90 0x7f4230141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.461+0000 7f4235e00640 1 --2- 172.21.15.47:0/1886485728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4230073040 0x7f42301411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.461+0000 7f4235e00640 1 --2- 172.21.15.47:0/1886485728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4230072c60 0x7f42300c2d60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.461+0000 7f4235e00640 1 -- 172.21.15.47:0/1886485728 >> 172.21.15.47:0/1886485728 conn(0x7f42300cb900 msgr2=0x7f42300cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.461+0000 7f4235e00640 1 -- 172.21.15.47:0/1886485728 shutdown_connections 2024-04-24T22:44:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.461+0000 7f4235e00640 1 -- 172.21.15.47:0/1886485728 wait complete. 2024-04-24T22:44:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.463+0000 7f4235e00640 1 Processor -- start 2024-04-24T22:44:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.463+0000 7f4235e00640 1 -- start start 2024-04-24T22:44:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f4235e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f423006bb90 0x7f42301ee090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f4235e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4230072c60 0x7f42300c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f4235e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4230073040 0x7f423013b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f4235e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f42300c32a0 con 0x7f423006bb90 2024-04-24T22:44:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f4235e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4230158270 con 0x7f4230072c60 2024-04-24T22:44:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f4235e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4230159080 con 0x7f4230073040 2024-04-24T22:44:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f422fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4230072c60 0x7f42300c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f4234a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f423006bb90 0x7f42301ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f422fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4230072c60 0x7f42300c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60134/0 (socket says 172.21.15.47:60134) 2024-04-24T22:44:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f422fe00640 1 -- 172.21.15.47:0/1692517516 learned_addr learned my addr 172.21.15.47:0/1692517516 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.464+0000 7f4235400640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4230073040 0x7f423013b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.465+0000 7f422fe00640 1 -- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4230073040 msgr2=0x7f423013b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.465+0000 7f422fe00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4230073040 0x7f423013b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.465+0000 7f422fe00640 1 -- 172.21.15.47:0/1692517516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f423006bb90 msgr2=0x7f42301ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.465+0000 7f422fe00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f423006bb90 0x7f42301ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.465+0000 7f422fe00640 1 -- 172.21.15.47:0/1692517516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423013a550 con 0x7f4230072c60 2024-04-24T22:44:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.465+0000 7f4234a00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f423006bb90 0x7f42301ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.465+0000 7f422fe00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4230072c60 0x7f42300c6dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f421c00e9b0 tx=0x7f421c00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.465+0000 7f422cc00640 1 -- 172.21.15.47:0/1692517516 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c017070 con 0x7f4230072c60 2024-04-24T22:44:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.466+0000 7f422cc00640 1 -- 172.21.15.47:0/1692517516 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421c00c9a0 con 0x7f4230072c60 2024-04-24T22:44:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.466+0000 7f422cc00640 1 -- 172.21.15.47:0/1692517516 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c00cd90 con 0x7f4230072c60 2024-04-24T22:44:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.466+0000 7f4235e00640 1 -- 172.21.15.47:0/1692517516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423013a830 con 0x7f4230072c60 2024-04-24T22:44:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.466+0000 7f4235e00640 1 -- 172.21.15.47:0/1692517516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423013adf0 con 0x7f4230072c60 2024-04-24T22:44:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.467+0000 7f4235e00640 1 -- 172.21.15.47:0/1692517516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42300ca560 con 0x7f4230072c60 2024-04-24T22:44:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.468+0000 7f422cc00640 1 -- 172.21.15.47:0/1692517516 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f421c01a070 con 0x7f4230072c60 2024-04-24T22:44:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.469+0000 7f422cc00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f421006be60 0x7f421006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.469+0000 7f422cc00640 1 -- 172.21.15.47:0/1692517516 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f421c08b2e0 con 0x7f4230072c60 2024-04-24T22:44:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.469+0000 7f4234a00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f421006be60 0x7f421006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.470+0000 7f4234a00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f421006be60 0x7f421006e320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4224034000 tx=0x7f422402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.473+0000 7f422cc00640 1 -- 172.21.15.47:0/1692517516 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f42300ca560 con 0x7f4230072c60 2024-04-24T22:44:06.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.674+0000 7f4235e00640 1 -- 172.21.15.47:0/1692517516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f423006b3a0 con 0x7f4230072c60 2024-04-24T22:44:06.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.675+0000 7f422cc00640 1 -- 172.21.15.47:0/1692517516 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f423006b3a0 con 0x7f4230072c60 2024-04-24T22:44:06.678 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-24T22:44:06.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.678+0000 7f4235e00640 1 -- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f421006be60 msgr2=0x7f421006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.678+0000 7f4235e00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f421006be60 0x7f421006e320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4224034000 tx=0x7f422402e000 comp rx=0 tx=0).stop 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.678+0000 7f4235e00640 1 -- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4230072c60 msgr2=0x7f42300c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.678+0000 7f4235e00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4230072c60 0x7f42300c6dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f421c00e9b0 tx=0x7f421c00ee70 comp rx=0 tx=0).stop 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.678+0000 7f4235e00640 1 -- 172.21.15.47:0/1692517516 shutdown_connections 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.678+0000 7f4235e00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f421006be60 0x7f421006e320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.678+0000 7f4235e00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4230073040 0x7f423013b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.678+0000 7f4235e00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4230072c60 0x7f42300c6dc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.678+0000 7f4235e00640 1 --2- 172.21.15.47:0/1692517516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f423006bb90 0x7f42301ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.678+0000 7f4235e00640 1 -- 172.21.15.47:0/1692517516 >> 172.21.15.47:0/1692517516 conn(0x7f42300cb900 msgr2=0x7f42300c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.679+0000 7f4235e00640 1 -- 172.21.15.47:0/1692517516 shutdown_connections 2024-04-24T22:44:06.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.679+0000 7f4235e00640 1 -- 172.21.15.47:0/1692517516 wait complete. 2024-04-24T22:44:06.703 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T22:44:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.914+0000 7efd8c000640 1 Processor -- start 2024-04-24T22:44:06.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.915+0000 7efd8c000640 1 -- start start 2024-04-24T22:44:06.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.917+0000 7efd8c000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd840c7750 0x7efd840c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.917+0000 7efd8c000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efd84072b80 0x7efd84072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.917+0000 7efd8c000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd840c7bb0 con 0x7efd84152960 2024-04-24T22:44:06.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.917+0000 7efd8c000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7efd84158f20 con 0x7efd84072b80 2024-04-24T22:44:06.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.917+0000 7efd8c000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efd841593b0 con 0x7efd840c7750 2024-04-24T22:44:06.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.917+0000 7efd89600640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efd84152960 0x7efd8415ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46588/0 (socket says 172.21.15.47:46588) 2024-04-24T22:44:06.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.917+0000 7efd89600640 1 -- 172.21.15.47:0/4186953524 learned_addr learned my addr 172.21.15.47:0/4186953524 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:06.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.917+0000 7efd88c00640 1 --2- 172.21.15.47:0/4186953524 >> v2:172.21.15.113:3300/0 conn(0x7efd840c7750 0x7efd840c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.917+0000 7efd8a000640 1 --2- 172.21.15.47:0/4186953524 >> v2:172.21.15.47:3300/0 conn(0x7efd84072b80 0x7efd84072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.918+0000 7efd88c00640 1 -- 172.21.15.47:0/4186953524 >> v2:172.21.15.47:3300/0 conn(0x7efd84072b80 msgr2=0x7efd84072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:06.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.918+0000 7efd88c00640 1 --2- 172.21.15.47:0/4186953524 >> v2:172.21.15.47:3300/0 conn(0x7efd84072b80 0x7efd84072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.918+0000 7efd88c00640 1 -- 172.21.15.47:0/4186953524 >> v1:172.21.15.47:6789/0 conn(0x7efd84152960 legacy=0x7efd8415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.918+0000 7efd88c00640 1 -- 172.21.15.47:0/4186953524 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd840c7e50 con 0x7efd840c7750 2024-04-24T22:44:06.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.919+0000 7efd88c00640 1 --2- 172.21.15.47:0/4186953524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840c7750 0x7efd840c8fa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efd84156080 tx=0x7efd7002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf3910877a10d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:06.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.919+0000 7efd83e00640 1 -- 172.21.15.47:0/4186953524 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70037070 con 0x7efd840c7750 2024-04-24T22:44:06.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.919+0000 7efd83e00640 1 -- 172.21.15.47:0/4186953524 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd70032440 con 0x7efd840c7750 2024-04-24T22:44:06.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.919+0000 7efd83e00640 1 -- 172.21.15.47:0/4186953524 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70032810 con 0x7efd840c7750 2024-04-24T22:44:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.920+0000 7efd8c000640 1 -- 172.21.15.47:0/4186953524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840c7750 msgr2=0x7efd840c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.920+0000 7efd8c000640 1 --2- 172.21.15.47:0/4186953524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840c7750 0x7efd840c8fa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efd84156080 tx=0x7efd7002a230 comp rx=0 tx=0).stop 2024-04-24T22:44:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.920+0000 7efd8c000640 1 -- 172.21.15.47:0/4186953524 shutdown_connections 2024-04-24T22:44:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.920+0000 7efd8c000640 1 --2- 172.21.15.47:0/4186953524 >> v2:172.21.15.47:3300/0 conn(0x7efd84072b80 0x7efd84072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.920+0000 7efd8c000640 1 --2- 172.21.15.47:0/4186953524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840c7750 0x7efd840c8fa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.920+0000 7efd8c000640 1 -- 172.21.15.47:0/4186953524 >> 172.21.15.47:0/4186953524 conn(0x7efd8406e580 msgr2=0x7efd8406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.920+0000 7efd8c000640 1 -- 172.21.15.47:0/4186953524 shutdown_connections 2024-04-24T22:44:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.920+0000 7efd8c000640 1 -- 172.21.15.47:0/4186953524 wait complete. 2024-04-24T22:44:06.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.922+0000 7efd8c000640 1 Processor -- start 2024-04-24T22:44:06.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.922+0000 7efd8c000640 1 -- start start 2024-04-24T22:44:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.923+0000 7efd8c000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd84072b80 0x7efd8414e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.923+0000 7efd8c000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd840c7750 0x7efd841f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.923+0000 7efd8c000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd84152960 0x7efd841f0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.923+0000 7efd8c000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efd841588d0 con 0x7efd84072b80 2024-04-24T22:44:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.923+0000 7efd8c000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd840c8130 con 0x7efd84152960 2024-04-24T22:44:06.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.923+0000 7efd8c000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efd84158bb0 con 0x7efd840c7750 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd89600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd84072b80 0x7efd8414e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd89600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd84072b80 0x7efd8414e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35570/0 (socket says 172.21.15.47:35570) 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd89600640 1 -- 172.21.15.47:0/967196403 learned_addr learned my addr 172.21.15.47:0/967196403 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd88c00640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd840c7750 0x7efd841f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd8a000640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd84152960 0x7efd841f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd89600640 1 -- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd840c7750 msgr2=0x7efd841f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd89600640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd840c7750 0x7efd841f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd89600640 1 -- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd84152960 msgr2=0x7efd841f0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd89600640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd84152960 0x7efd841f0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd89600640 1 -- 172.21.15.47:0/967196403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd840c3930 con 0x7efd84072b80 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.924+0000 7efd88c00640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd840c7750 0x7efd841f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.925+0000 7efd89600640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd84072b80 0x7efd8414e8a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efd7800caa0 tx=0x7efd7800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:06.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.925+0000 7efd81600640 1 -- 172.21.15.47:0/967196403 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78017070 con 0x7efd84072b80 2024-04-24T22:44:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.925+0000 7efd81600640 1 -- 172.21.15.47:0/967196403 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7800ac20 con 0x7efd84072b80 2024-04-24T22:44:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.925+0000 7efd81600640 1 -- 172.21.15.47:0/967196403 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78012400 con 0x7efd84072b80 2024-04-24T22:44:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.925+0000 7efd8c000640 1 -- 172.21.15.47:0/967196403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd840b09b0 con 0x7efd84072b80 2024-04-24T22:44:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.925+0000 7efd8c000640 1 -- 172.21.15.47:0/967196403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd840b0f70 con 0x7efd84072b80 2024-04-24T22:44:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.926+0000 7efd8c000640 1 -- 172.21.15.47:0/967196403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd84073b90 con 0x7efd84072b80 2024-04-24T22:44:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.927+0000 7efd81600640 1 -- 172.21.15.47:0/967196403 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd7801a070 con 0x7efd84072b80 2024-04-24T22:44:06.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.929+0000 7efd81600640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd6406be30 0x7efd6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:06.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.929+0000 7efd81600640 1 -- 172.21.15.47:0/967196403 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7efd7808ab00 con 0x7efd84072b80 2024-04-24T22:44:06.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.930+0000 7efd88c00640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd6406be30 0x7efd6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:06.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.930+0000 7efd88c00640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd6406be30 0x7efd6406e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efd70034000 tx=0x7efd7002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:06.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:06.935+0000 7efd81600640 1 -- 172.21.15.47:0/967196403 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd78058d40 con 0x7efd84072b80 2024-04-24T22:44:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.151+0000 7efd8c000640 1 -- 172.21.15.47:0/967196403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efd8415c8c0 con 0x7efd84072b80 2024-04-24T22:44:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.152+0000 7efd81600640 1 -- 172.21.15.47:0/967196403 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7efd7808ae20 con 0x7efd84072b80 2024-04-24T22:44:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-24T22:44:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.155+0000 7efd8c000640 1 -- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd6406be30 msgr2=0x7efd6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.155+0000 7efd8c000640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd6406be30 0x7efd6406e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efd70034000 tx=0x7efd7002e000 comp rx=0 tx=0).stop 2024-04-24T22:44:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.155+0000 7efd8c000640 1 -- 172.21.15.47:0/967196403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd84072b80 msgr2=0x7efd8414e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.155+0000 7efd8c000640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd84072b80 0x7efd8414e8a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efd7800caa0 tx=0x7efd7800cf60 comp rx=0 tx=0).stop 2024-04-24T22:44:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.155+0000 7efd8c000640 1 -- 172.21.15.47:0/967196403 shutdown_connections 2024-04-24T22:44:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.155+0000 7efd8c000640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7efd6406be30 0x7efd6406e2f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.155+0000 7efd8c000640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd84152960 0x7efd841f0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.155+0000 7efd8c000640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd840c7750 0x7efd841f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.155+0000 7efd8c000640 1 --2- 172.21.15.47:0/967196403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd84072b80 0x7efd8414e8a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.155+0000 7efd8c000640 1 -- 172.21.15.47:0/967196403 >> 172.21.15.47:0/967196403 conn(0x7efd8406e580 msgr2=0x7efd840c0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.156+0000 7efd8c000640 1 -- 172.21.15.47:0/967196403 shutdown_connections 2024-04-24T22:44:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.156+0000 7efd8c000640 1 -- 172.21.15.47:0/967196403 wait complete. 2024-04-24T22:44:07.179 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T22:44:07.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:44:07.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T22:44:07.311 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:44:07.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:44:07.312 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T22:44:07.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:44:07.373 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-24T22:44:07.412 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T22:44:07.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:44:07.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T22:44:07.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T22:44:07.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-24T22:44:07.637 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-24T22:44:07.651 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T22:44:07.651 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:44:07.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.831+0000 7f1530600640 1 Processor -- start 2024-04-24T22:44:07.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.831+0000 7f1530600640 1 -- start start 2024-04-24T22:44:07.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.833+0000 7f1530600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1528154960 0x7f152815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:07.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.833+0000 7f1530600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f152815ab10 con 0x7f152815e830 2024-04-24T22:44:07.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.833+0000 7f1530600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f152815adf0 con 0x7f152815a680 2024-04-24T22:44:07.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.833+0000 7f1530600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f152815b0d0 con 0x7f1528154960 2024-04-24T22:44:07.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.833+0000 7f152dc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1528154960 0x7f152815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:07.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.833+0000 7f152dc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1528154960 0x7f152815e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35574/0 (socket says 172.21.15.47:35574) 2024-04-24T22:44:07.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.833+0000 7f152dc00640 1 -- 172.21.15.47:0/4207396739 learned_addr learned my addr 172.21.15.47:0/4207396739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:07.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.834+0000 7f152dc00640 1 -- 172.21.15.47:0/4207396739 >> v1:172.21.15.113:6789/0 conn(0x7f152815a680 legacy=0x7f15280c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:07.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.834+0000 7f152dc00640 1 -- 172.21.15.47:0/4207396739 >> v1:172.21.15.47:6789/0 conn(0x7f152815e830 legacy=0x7f15280c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:07.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.834+0000 7f152dc00640 1 -- 172.21.15.47:0/4207396739 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152815b3b0 con 0x7f1528154960 2024-04-24T22:44:07.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.835+0000 7f152dc00640 1 --2- 172.21.15.47:0/4207396739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 0x7f152815e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f151c009e50 tx=0x7f151c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3135f89cd8052d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:07.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.835+0000 7f1527e00640 1 -- 172.21.15.47:0/4207396739 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c036070 con 0x7f1528154960 2024-04-24T22:44:07.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.835+0000 7f1527e00640 1 -- 172.21.15.47:0/4207396739 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151c031600 con 0x7f1528154960 2024-04-24T22:44:07.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.835+0000 7f1527e00640 1 -- 172.21.15.47:0/4207396739 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0319f0 con 0x7f1528154960 2024-04-24T22:44:07.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.836+0000 7f1530600640 1 -- 172.21.15.47:0/4207396739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 msgr2=0x7f152815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:07.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.836+0000 7f1530600640 1 --2- 172.21.15.47:0/4207396739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 0x7f152815e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f151c009e50 tx=0x7f151c02a7c0 comp rx=0 tx=0).stop 2024-04-24T22:44:07.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.836+0000 7f1530600640 1 -- 172.21.15.47:0/4207396739 shutdown_connections 2024-04-24T22:44:07.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.836+0000 7f1530600640 1 --2- 172.21.15.47:0/4207396739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 0x7f152815e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.836+0000 7f1530600640 1 -- 172.21.15.47:0/4207396739 >> 172.21.15.47:0/4207396739 conn(0x7f152806e580 msgr2=0x7f152806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.836+0000 7f1530600640 1 -- 172.21.15.47:0/4207396739 shutdown_connections 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.837+0000 7f1530600640 1 -- 172.21.15.47:0/4207396739 wait complete. 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.838+0000 7f1530600640 1 Processor -- start 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.838+0000 7f1530600640 1 -- start start 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.839+0000 7f1530600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 0x7f15281f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.839+0000 7f1530600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f152815e830 0x7f15281f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.839+0000 7f1530600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15281e3a90 0x7f152814cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.839+0000 7f1530600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f15280c8ca0 con 0x7f1528154960 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.839+0000 7f1530600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1528154d40 con 0x7f15281e3a90 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.839+0000 7f1530600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f15280c8e20 con 0x7f152815e830 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 0x7f15281f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 0x7f15281f1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35586/0 (socket says 172.21.15.47:35586) 2024-04-24T22:44:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152dc00640 1 -- 172.21.15.47:0/1137274163 learned_addr learned my addr 172.21.15.47:0/1137274163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:07.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152d200640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f152815e830 0x7f15281f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:07.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152e600640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15281e3a90 0x7f152814cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:07.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152dc00640 1 -- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f152815e830 msgr2=0x7f15281f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:07.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152dc00640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f152815e830 0x7f15281f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:07.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152dc00640 1 -- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15281e3a90 msgr2=0x7f152814cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:07.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152dc00640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15281e3a90 0x7f152814cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:07.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152dc00640 1 -- 172.21.15.47:0/1137274163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1528157070 con 0x7f1528154960 2024-04-24T22:44:07.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152d200640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f152815e830 0x7f15281f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:44:07.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152e600640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15281e3a90 0x7f152814cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:44:07.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.840+0000 7f152dc00640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 0x7f15281f1850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f151c009f70 tx=0x7f151c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:07.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.841+0000 7f1525600640 1 -- 172.21.15.47:0/1137274163 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c036050 con 0x7f1528154960 2024-04-24T22:44:07.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.841+0000 7f1525600640 1 -- 172.21.15.47:0/1137274163 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151c042070 con 0x7f1528154960 2024-04-24T22:44:07.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.841+0000 7f1525600640 1 -- 172.21.15.47:0/1137274163 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c009510 con 0x7f1528154960 2024-04-24T22:44:07.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.841+0000 7f1530600640 1 -- 172.21.15.47:0/1137274163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1528157350 con 0x7f1528154960 2024-04-24T22:44:07.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.841+0000 7f1530600640 1 -- 172.21.15.47:0/1137274163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1528157910 con 0x7f1528154960 2024-04-24T22:44:07.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.842+0000 7f1530600640 1 -- 172.21.15.47:0/1137274163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1528157fd0 con 0x7f1528154960 2024-04-24T22:44:07.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.843+0000 7f1525600640 1 -- 172.21.15.47:0/1137274163 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f151c004470 con 0x7f1528154960 2024-04-24T22:44:07.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.844+0000 7f1525600640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f150806be30 0x7f150806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:07.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.844+0000 7f1525600640 1 -- 172.21.15.47:0/1137274163 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f151c0abee0 con 0x7f1528154960 2024-04-24T22:44:07.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.844+0000 7f152d200640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f150806be30 0x7f150806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:07.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.845+0000 7f152d200640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f150806be30 0x7f150806e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1514007fd0 tx=0x7f1514009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:07.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:07.849+0000 7f1525600640 1 -- 172.21.15.47:0/1137274163 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1528157fd0 con 0x7f1528154960 2024-04-24T22:44:08.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.049+0000 7f1530600640 1 -- 172.21.15.47:0/1137274163 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1528156e20 con 0x7f150806be30 2024-04-24T22:44:08.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.056+0000 7f1525600640 1 -- 172.21.15.47:0/1137274163 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252947 (secure 0 0 0) 0x7f1528156e20 con 0x7f150806be30 2024-04-24T22:44:08.064 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:44:08.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 -- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f150806be30 msgr2=0x7f150806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f150806be30 0x7f150806e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1514007fd0 tx=0x7f1514009040 comp rx=0 tx=0).stop 2024-04-24T22:44:08.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 -- 172.21.15.47:0/1137274163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 msgr2=0x7f15281f1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 0x7f15281f1850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f151c009f70 tx=0x7f151c004a40 comp rx=0 tx=0).stop 2024-04-24T22:44:08.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 -- 172.21.15.47:0/1137274163 shutdown_connections 2024-04-24T22:44:08.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f150806be30 0x7f150806e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15281e3a90 0x7f152814cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f152815e830 0x7f15281f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 --2- 172.21.15.47:0/1137274163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1528154960 0x7f15281f1850 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 -- 172.21.15.47:0/1137274163 >> 172.21.15.47:0/1137274163 conn(0x7f152806e580 msgr2=0x7f1528159bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:08.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 -- 172.21.15.47:0/1137274163 shutdown_connections 2024-04-24T22:44:08.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.062+0000 7f1530600640 1 -- 172.21.15.47:0/1137274163 wait complete. 2024-04-24T22:44:08.085 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-24T22:44:06.703050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":981383,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":9788,"num_write_kb":25168823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11246,"ondisk_log_size":11246,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282584,"kb_used_data":4056,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754692136,"statfs":{"total":773094113280,"available":772804747264,"internally_reserved":0,"allocated":4153344,"data_stored":2665126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":285134387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9425,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003007"},"pg_stats":[{"pgid":"3.3f","version":"23'159","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.666893+0000","last_change":"2024-04-24T22:40:23.914828+0000","last_active":"2024-04-24T22:43:48.666893+0000","last_peered":"2024-04-24T22:43:48.666893+0000","last_clean":"2024-04-24T22:43:48.666893+0000","last_became_active":"2024-04-24T22:40:23.913848+0000","last_became_peered":"2024-04-24T22:40:23.913848+0000","last_unstale":"2024-04-24T22:43:48.666893+0000","last_undegraded":"2024-04-24T22:43:48.666893+0000","last_fullsized":"2024-04-24T22:43:48.666893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.690809+0000","last_change":"2024-04-24T22:39:37.697245+0000","last_active":"2024-04-24T22:43:48.690809+0000","last_peered":"2024-04-24T22:43:48.690809+0000","last_clean":"2024-04-24T22:43:48.690809+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:43:48.690809+0000","last_undegraded":"2024-04-24T22:43:48.690809+0000","last_fullsized":"2024-04-24T22:43:48.690809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.689091+0000","last_change":"2024-04-24T22:39:37.709539+0000","last_active":"2024-04-24T22:43:48.689091+0000","last_peered":"2024-04-24T22:43:48.689091+0000","last_clean":"2024-04-24T22:43:48.689091+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:43:48.689091+0000","last_undegraded":"2024-04-24T22:43:48.689091+0000","last_fullsized":"2024-04-24T22:43:48.689091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'185","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.691701+0000","last_change":"2024-04-24T22:39:37.719136+0000","last_active":"2024-04-24T22:43:48.691701+0000","last_peered":"2024-04-24T22:43:48.691701+0000","last_clean":"2024-04-24T22:43:48.691701+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:43:48.691701+0000","last_undegraded":"2024-04-24T22:43:48.691701+0000","last_fullsized":"2024-04-24T22:43:48.691701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.667187+0000","last_change":"2024-04-24T22:39:37.704671+0000","last_active":"2024-04-24T22:43:48.667187+0000","last_peered":"2024-04-24T22:43:48.667187+0000","last_clean":"2024-04-24T22:43:48.667187+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:43:48.667187+0000","last_undegraded":"2024-04-24T22:43:48.667187+0000","last_fullsized":"2024-04-24T22:43:48.667187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.683504+0000","last_change":"2024-04-24T22:39:37.703880+0000","last_active":"2024-04-24T22:43:48.683504+0000","last_peered":"2024-04-24T22:43:48.683504+0000","last_clean":"2024-04-24T22:43:48.683504+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:43:48.683504+0000","last_undegraded":"2024-04-24T22:43:48.683504+0000","last_fullsized":"2024-04-24T22:43:48.683504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.627153+0000","last_change":"2024-04-24T22:39:37.703539+0000","last_active":"2024-04-24T22:43:48.627153+0000","last_peered":"2024-04-24T22:43:48.627153+0000","last_clean":"2024-04-24T22:43:48.627153+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:43:48.627153+0000","last_undegraded":"2024-04-24T22:43:48.627153+0000","last_fullsized":"2024-04-24T22:43:48.627153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'195","reported_seq":212,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.682546+0000","last_change":"2024-04-24T22:39:37.713543+0000","last_active":"2024-04-24T22:43:48.682546+0000","last_peered":"2024-04-24T22:43:48.682546+0000","last_clean":"2024-04-24T22:43:48.682546+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:43:48.682546+0000","last_undegraded":"2024-04-24T22:43:48.682546+0000","last_fullsized":"2024-04-24T22:43:48.682546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'167","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.645645+0000","last_change":"2024-04-24T22:39:37.706715+0000","last_active":"2024-04-24T22:43:48.645645+0000","last_peered":"2024-04-24T22:43:48.645645+0000","last_clean":"2024-04-24T22:43:48.645645+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:43:48.645645+0000","last_undegraded":"2024-04-24T22:43:48.645645+0000","last_fullsized":"2024-04-24T22:43:48.645645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'175","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.672260+0000","last_change":"2024-04-24T22:40:23.915517+0000","last_active":"2024-04-24T22:43:48.672260+0000","last_peered":"2024-04-24T22:43:48.672260+0000","last_clean":"2024-04-24T22:43:48.672260+0000","last_became_active":"2024-04-24T22:40:23.915141+0000","last_became_peered":"2024-04-24T22:40:23.915141+0000","last_unstale":"2024-04-24T22:43:48.672260+0000","last_undegraded":"2024-04-24T22:43:48.672260+0000","last_fullsized":"2024-04-24T22:43:48.672260+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:07.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'208","reported_seq":225,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.677013+0000","last_change":"2024-04-24T22:39:37.701069+0000","last_active":"2024-04-24T22:43:48.677013+0000","last_peered":"2024-04-24T22:43:48.677013+0000","last_clean":"2024-04-24T22:43:48.677013+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:43:48.677013+0000","last_undegraded":"2024-04-24T22:43:48.677013+0000","last_fullsized":"2024-04-24T22:43:48.677013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.686235+0000","last_change":"2024-04-24T22:39:37.693508+0000","last_active":"2024-04-24T22:43:48.686235+0000","last_peered":"2024-04-24T22:43:48.686235+0000","last_clean":"2024-04-24T22:43:48.686235+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:43:48.686235+0000","last_undegraded":"2024-04-24T22:43:48.686235+0000","last_fullsized":"2024-04-24T22:43:48.686235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'167","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.669776+0000","last_change":"2024-04-24T22:39:37.705157+0000","last_active":"2024-04-24T22:43:48.669776+0000","last_peered":"2024-04-24T22:43:48.669776+0000","last_clean":"2024-04-24T22:43:48.669776+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:43:48.669776+0000","last_undegraded":"2024-04-24T22:43:48.669776+0000","last_fullsized":"2024-04-24T22:43:48.669776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'164","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.622338+0000","last_change":"2024-04-24T22:39:37.698333+0000","last_active":"2024-04-24T22:43:48.622338+0000","last_peered":"2024-04-24T22:43:48.622338+0000","last_clean":"2024-04-24T22:43:48.622338+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:43:48.622338+0000","last_undegraded":"2024-04-24T22:43:48.622338+0000","last_fullsized":"2024-04-24T22:43:48.622338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:57.510197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.671167+0000","last_change":"2024-04-24T22:39:37.696874+0000","last_active":"2024-04-24T22:43:48.671167+0000","last_peered":"2024-04-24T22:43:48.671167+0000","last_clean":"2024-04-24T22:43:48.671167+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:43:48.671167+0000","last_undegraded":"2024-04-24T22:43:48.671167+0000","last_fullsized":"2024-04-24T22:43:48.671167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.654424+0000","last_change":"2024-04-24T22:39:37.703446+0000","last_active":"2024-04-24T22:43:48.654424+0000","last_peered":"2024-04-24T22:43:48.654424+0000","last_clean":"2024-04-24T22:43:48.654424+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:43:48.654424+0000","last_undegraded":"2024-04-24T22:43:48.654424+0000","last_fullsized":"2024-04-24T22:43:48.654424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'168","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.666782+0000","last_change":"2024-04-24T22:40:23.916129+0000","last_active":"2024-04-24T22:43:48.666782+0000","last_peered":"2024-04-24T22:43:48.666782+0000","last_clean":"2024-04-24T22:43:48.666782+0000","last_became_active":"2024-04-24T22:40:23.915441+0000","last_became_peered":"2024-04-24T22:40:23.915441+0000","last_unstale":"2024-04-24T22:43:48.666782+0000","last_undegraded":"2024-04-24T22:43:48.666782+0000","last_fullsized":"2024-04-24T22:43:48.666782+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'169","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.672272+0000","last_change":"2024-04-24T22:39:37.703899+0000","last_active":"2024-04-24T22:43:48.672272+0000","last_peered":"2024-04-24T22:43:48.672272+0000","last_clean":"2024-04-24T22:43:48.672272+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:43:48.672272+0000","last_undegraded":"2024-04-24T22:43:48.672272+0000","last_fullsized":"2024-04-24T22:43:48.672272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'198","reported_seq":215,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.680661+0000","last_change":"2024-04-24T22:39:37.711019+0000","last_active":"2024-04-24T22:43:48.680661+0000","last_peered":"2024-04-24T22:43:48.680661+0000","last_clean":"2024-04-24T22:43:48.680661+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:43:48.680661+0000","last_undegraded":"2024-04-24T22:43:48.680661+0000","last_fullsized":"2024-04-24T22:43:48.680661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'171","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.685173+0000","last_change":"2024-04-24T22:39:37.696214+0000","last_active":"2024-04-24T22:43:48.685173+0000","last_peered":"2024-04-24T22:43:48.685173+0000","last_clean":"2024-04-24T22:43:48.685173+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:43:48.685173+0000","last_undegraded":"2024-04-24T22:43:48.685173+0000","last_fullsized":"2024-04-24T22:43:48.685173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'143","reported_seq":160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.654913+0000","last_change":"2024-04-24T22:39:37.700151+0000","last_active":"2024-04-24T22:43:48.654913+0000","last_peered":"2024-04-24T22:43:48.654913+0000","last_clean":"2024-04-24T22:43:48.654913+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:43:48.654913+0000","last_undegraded":"2024-04-24T22:43:48.654913+0000","last_fullsized":"2024-04-24T22:43:48.654913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'189","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.664027+0000","last_change":"2024-04-24T22:39:37.705083+0000","last_active":"2024-04-24T22:43:48.664027+0000","last_peered":"2024-04-24T22:43:48.664027+0000","last_clean":"2024-04-24T22:43:48.664027+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:43:48.664027+0000","last_undegraded":"2024-04-24T22:43:48.664027+0000","last_fullsized":"2024-04-24T22:43:48.664027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'142","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.652343+0000","last_change":"2024-04-24T22:39:37.711193+0000","last_active":"2024-04-24T22:43:48.652343+0000","last_peered":"2024-04-24T22:43:48.652343+0000","last_clean":"2024-04-24T22:43:48.652343+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:43:48.652343+0000","last_undegraded":"2024-04-24T22:43:48.652343+0000","last_fullsized":"2024-04-24T22:43:48.652343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'201","reported_seq":219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.685527+0000","last_change":"2024-04-24T22:39:37.713910+0000","last_active":"2024-04-24T22:43:48.685527+0000","last_peered":"2024-04-24T22:43:48.685527+0000","last_clean":"2024-04-24T22:43:48.685527+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:43:48.685527+0000","last_undegraded":"2024-04-24T22:43:48.685527+0000","last_fullsized":"2024-04-24T22:43:48.685527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'158","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.666804+0000","last_change":"2024-04-24T22:39:37.717191+0000","last_active":"2024-04-24T22:43:48.666804+0000","last_peered":"2024-04-24T22:43:48.666804+0000","last_clean":"2024-04-24T22:43:48.666804+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:43:48.666804+0000","last_undegraded":"2024-04-24T22:43:48.666804+0000","last_fullsized":"2024-04-24T22:43:48.666804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'202","reported_seq":220,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.638120+0000","last_change":"2024-04-24T22:39:37.702983+0000","last_active":"2024-04-24T22:43:48.638120+0000","last_peered":"2024-04-24T22:43:48.638120+0000","last_clean":"2024-04-24T22:43:48.638120+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:43:48.638120+0000","last_undegraded":"2024-04-24T22:43:48.638120+0000","last_fullsized":"2024-04-24T22:43:48.638120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.682048+0000","last_change":"2024-04-24T22:39:37.714111+0000","last_active":"2024-04-24T22:43:48.682048+0000","last_peered":"2024-04-24T22:43:48.682048+0000","last_clean":"2024-04-24T22:43:48.682048+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:43:48.682048+0000","last_undegraded":"2024-04-24T22:43:48.682048+0000","last_fullsized":"2024-04-24T22:43:48.682048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.669969+0000","last_change":"2024-04-24T22:39:37.714877+0000","last_active":"2024-04-24T22:43:48.669969+0000","last_peered":"2024-04-24T22:43:48.669969+0000","last_clean":"2024-04-24T22:43:48.669969+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:43:48.669969+0000","last_undegraded":"2024-04-24T22:43:48.669969+0000","last_fullsized":"2024-04-24T22:43:48.669969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'127","reported_seq":145,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.674841+0000","last_change":"2024-04-24T22:39:37.714695+0000","last_active":"2024-04-24T22:43:48.674841+0000","last_peered":"2024-04-24T22:43:48.674841+0000","last_clean":"2024-04-24T22:43:48.674841+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:43:48.674841+0000","last_undegraded":"2024-04-24T22:43:48.674841+0000","last_fullsized":"2024-04-24T22:43:48.674841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'145","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.656139+0000","last_change":"2024-04-24T22:39:37.719187+0000","last_active":"2024-04-24T22:43:48.656139+0000","last_peered":"2024-04-24T22:43:48.656139+0000","last_clean":"2024-04-24T22:43:48.656139+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:43:48.656139+0000","last_undegraded":"2024-04-24T22:43:48.656139+0000","last_fullsized":"2024-04-24T22:43:48.656139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'146","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.688744+0000","last_change":"2024-04-24T22:39:37.707283+0000","last_active":"2024-04-24T22:43:48.688744+0000","last_peered":"2024-04-24T22:43:48.688744+0000","last_clean":"2024-04-24T22:43:48.688744+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:43:48.688744+0000","last_undegraded":"2024-04-24T22:43:48.688744+0000","last_fullsized":"2024-04-24T22:43:48.688744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.683443+0000","last_change":"2024-04-24T22:39:37.697902+0000","last_active":"2024-04-24T22:43:48.683443+0000","last_peered":"2024-04-24T22:43:48.683443+0000","last_clean":"2024-04-24T22:43:48.683443+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:43:48.683443+0000","last_undegraded":"2024-04-24T22:43:48.683443+0000","last_fullsized":"2024-04-24T22:43:48.683443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'142","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.654469+0000","last_change":"2024-04-24T22:39:37.705667+0000","last_active":"2024-04-24T22:43:48.654469+0000","last_peered":"2024-04-24T22:43:48.654469+0000","last_clean":"2024-04-24T22:43:48.654469+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:43:48.654469+0000","last_undegraded":"2024-04-24T22:43:48.654469+0000","last_fullsized":"2024-04-24T22:43:48.654469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:50.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:24.451020+0000","last_change":"2024-04-24T22:39:36.707990+0000","last_active":"2024-04-24T22:40:24.451020+0000","last_peered":"2024-04-24T22:40:24.451020+0000","last_clean":"2024-04-24T22:40:24.451020+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:40:24.451020+0000","last_undegraded":"2024-04-24T22:40:24.451020+0000","last_fullsized":"2024-04-24T22:40:24.451020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'154","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.622757+0000","last_change":"2024-04-24T22:39:37.700406+0000","last_active":"2024-04-24T22:43:48.622757+0000","last_peered":"2024-04-24T22:43:48.622757+0000","last_clean":"2024-04-24T22:43:48.622757+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:43:48.622757+0000","last_undegraded":"2024-04-24T22:43:48.622757+0000","last_fullsized":"2024-04-24T22:43:48.622757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.908485+0000","last_change":"2024-04-24T22:39:36.701398+0000","last_active":"2024-04-24T22:40:23.908485+0000","last_peered":"2024-04-24T22:40:23.908485+0000","last_clean":"2024-04-24T22:40:23.908485+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:40:23.908485+0000","last_undegraded":"2024-04-24T22:40:23.908485+0000","last_fullsized":"2024-04-24T22:40:23.908485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:24.450888+0000","last_change":"2024-04-24T22:39:36.712294+0000","last_active":"2024-04-24T22:40:24.450888+0000","last_peered":"2024-04-24T22:40:24.450888+0000","last_clean":"2024-04-24T22:40:24.450888+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:40:24.450888+0000","last_undegraded":"2024-04-24T22:40:24.450888+0000","last_fullsized":"2024-04-24T22:40:24.450888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'182","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673441+0000","last_change":"2024-04-24T22:40:23.910148+0000","last_active":"2024-04-24T22:43:48.673441+0000","last_peered":"2024-04-24T22:43:48.673441+0000","last_clean":"2024-04-24T22:43:48.673441+0000","last_became_active":"2024-04-24T22:40:23.909573+0000","last_became_peered":"2024-04-24T22:40:23.909573+0000","last_unstale":"2024-04-24T22:43:48.673441+0000","last_undegraded":"2024-04-24T22:43:48.673441+0000","last_fullsized":"2024-04-24T22:43:48.673441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:07.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906152+0000","last_change":"2024-04-24T22:39:36.698983+0000","last_active":"2024-04-24T22:40:23.906152+0000","last_peered":"2024-04-24T22:40:23.906152+0000","last_clean":"2024-04-24T22:40:23.906152+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:40:23.906152+0000","last_undegraded":"2024-04-24T22:40:23.906152+0000","last_fullsized":"2024-04-24T22:40:23.906152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'177","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.676625+0000","last_change":"2024-04-24T22:39:37.714229+0000","last_active":"2024-04-24T22:43:48.676625+0000","last_peered":"2024-04-24T22:43:48.676625+0000","last_clean":"2024-04-24T22:43:48.676625+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:43:48.676625+0000","last_undegraded":"2024-04-24T22:43:48.676625+0000","last_fullsized":"2024-04-24T22:43:48.676625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:04.691585+0000","last_change":"2024-04-24T22:39:36.705776+0000","last_active":"2024-04-24T22:42:04.691585+0000","last_peered":"2024-04-24T22:42:04.691585+0000","last_clean":"2024-04-24T22:42:04.691585+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:42:04.691585+0000","last_undegraded":"2024-04-24T22:42:04.691585+0000","last_fullsized":"2024-04-24T22:42:04.691585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'241","reported_seq":258,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673855+0000","last_change":"2024-04-24T22:39:37.701436+0000","last_active":"2024-04-24T22:43:48.673855+0000","last_peered":"2024-04-24T22:43:48.673855+0000","last_clean":"2024-04-24T22:43:48.673855+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:43:48.673855+0000","last_undegraded":"2024-04-24T22:43:48.673855+0000","last_fullsized":"2024-04-24T22:43:48.673855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906331+0000","last_change":"2024-04-24T22:39:36.700237+0000","last_active":"2024-04-24T22:40:23.906331+0000","last_peered":"2024-04-24T22:40:23.906331+0000","last_clean":"2024-04-24T22:40:23.906331+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:40:23.906331+0000","last_undegraded":"2024-04-24T22:40:23.906331+0000","last_fullsized":"2024-04-24T22:40:23.906331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'151","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.660220+0000","last_change":"2024-04-24T22:39:37.703532+0000","last_active":"2024-04-24T22:43:48.660220+0000","last_peered":"2024-04-24T22:43:48.660220+0000","last_clean":"2024-04-24T22:43:48.660220+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:43:48.660220+0000","last_undegraded":"2024-04-24T22:43:48.660220+0000","last_fullsized":"2024-04-24T22:43:48.660220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'11","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:53.085629+0000","last_change":"2024-04-24T22:39:36.717477+0000","last_active":"2024-04-24T22:43:53.085629+0000","last_peered":"2024-04-24T22:43:53.085629+0000","last_clean":"2024-04-24T22:43:53.085629+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:43:53.085629+0000","last_undegraded":"2024-04-24T22:43:53.085629+0000","last_fullsized":"2024-04-24T22:43:53.085629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'157","reported_seq":174,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.640565+0000","last_change":"2024-04-24T22:39:37.699255+0000","last_active":"2024-04-24T22:43:48.640565+0000","last_peered":"2024-04-24T22:43:48.640565+0000","last_clean":"2024-04-24T22:43:48.640565+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:43:48.640565+0000","last_undegraded":"2024-04-24T22:43:48.640565+0000","last_fullsized":"2024-04-24T22:43:48.640565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'166","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:53.750307+0000","last_change":"2024-04-24T22:39:36.706567+0000","last_active":"2024-04-24T22:43:53.750307+0000","last_peered":"2024-04-24T22:43:53.750307+0000","last_clean":"2024-04-24T22:43:53.750307+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:43:53.750307+0000","last_undegraded":"2024-04-24T22:43:53.750307+0000","last_fullsized":"2024-04-24T22:43:53.750307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383830,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.670888+0000","last_change":"2024-04-24T22:39:37.712064+0000","last_active":"2024-04-24T22:43:48.670888+0000","last_peered":"2024-04-24T22:43:48.670888+0000","last_clean":"2024-04-24T22:43:48.670888+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:43:48.670888+0000","last_undegraded":"2024-04-24T22:43:48.670888+0000","last_fullsized":"2024-04-24T22:43:48.670888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'9","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.903984+0000","last_change":"2024-04-24T22:39:36.700041+0000","last_active":"2024-04-24T22:40:23.903984+0000","last_peered":"2024-04-24T22:40:23.903984+0000","last_clean":"2024-04-24T22:40:23.903984+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:40:23.903984+0000","last_undegraded":"2024-04-24T22:40:23.903984+0000","last_fullsized":"2024-04-24T22:40:23.903984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:50.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'168","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.624474+0000","last_change":"2024-04-24T22:39:37.691459+0000","last_active":"2024-04-24T22:43:48.624474+0000","last_peered":"2024-04-24T22:43:48.624474+0000","last_clean":"2024-04-24T22:43:48.624474+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:43:48.624474+0000","last_undegraded":"2024-04-24T22:43:48.624474+0000","last_fullsized":"2024-04-24T22:43:48.624474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906390+0000","last_change":"2024-04-24T22:39:36.706454+0000","last_active":"2024-04-24T22:40:23.906390+0000","last_peered":"2024-04-24T22:40:23.906390+0000","last_clean":"2024-04-24T22:40:23.906390+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:40:23.906390+0000","last_undegraded":"2024-04-24T22:40:23.906390+0000","last_fullsized":"2024-04-24T22:40:23.906390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'186","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.675788+0000","last_change":"2024-04-24T22:39:37.696724+0000","last_active":"2024-04-24T22:43:48.675788+0000","last_peered":"2024-04-24T22:43:48.675788+0000","last_clean":"2024-04-24T22:43:48.675788+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:43:48.675788+0000","last_undegraded":"2024-04-24T22:43:48.675788+0000","last_fullsized":"2024-04-24T22:43:48.675788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.908547+0000","last_change":"2024-04-24T22:39:36.701340+0000","last_active":"2024-04-24T22:40:23.908547+0000","last_peered":"2024-04-24T22:40:23.908547+0000","last_clean":"2024-04-24T22:40:23.908547+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:40:23.908547+0000","last_undegraded":"2024-04-24T22:40:23.908547+0000","last_fullsized":"2024-04-24T22:40:23.908547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'191","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.679977+0000","last_change":"2024-04-24T22:39:37.697532+0000","last_active":"2024-04-24T22:43:48.679977+0000","last_peered":"2024-04-24T22:43:48.679977+0000","last_clean":"2024-04-24T22:43:48.679977+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:43:48.679977+0000","last_undegraded":"2024-04-24T22:43:48.679977+0000","last_fullsized":"2024-04-24T22:43:48.679977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'18","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:59.706381+0000","last_change":"2024-04-24T22:39:36.724649+0000","last_active":"2024-04-24T22:41:59.706381+0000","last_peered":"2024-04-24T22:41:59.706381+0000","last_clean":"2024-04-24T22:41:59.706381+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:41:59.706381+0000","last_undegraded":"2024-04-24T22:41:59.706381+0000","last_fullsized":"2024-04-24T22:41:59.706381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'209","reported_seq":226,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.637954+0000","last_change":"2024-04-24T22:39:37.702890+0000","last_active":"2024-04-24T22:43:48.637954+0000","last_peered":"2024-04-24T22:43:48.637954+0000","last_clean":"2024-04-24T22:43:48.637954+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:43:48.637954+0000","last_undegraded":"2024-04-24T22:43:48.637954+0000","last_fullsized":"2024-04-24T22:43:48.637954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:24.792909+0000","last_change":"2024-04-24T22:39:36.716044+0000","last_active":"2024-04-24T22:40:24.792909+0000","last_peered":"2024-04-24T22:40:24.792909+0000","last_clean":"2024-04-24T22:40:24.792909+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:40:24.792909+0000","last_undegraded":"2024-04-24T22:40:24.792909+0000","last_fullsized":"2024-04-24T22:40:24.792909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:40.023933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.692442+0000","last_change":"2024-04-24T22:39:37.702621+0000","last_active":"2024-04-24T22:43:48.692442+0000","last_peered":"2024-04-24T22:43:48.692442+0000","last_clean":"2024-04-24T22:43:48.692442+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:43:48.692442+0000","last_undegraded":"2024-04-24T22:43:48.692442+0000","last_fullsized":"2024-04-24T22:43:48.692442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.915496+0000","last_change":"2024-04-24T22:39:36.714272+0000","last_active":"2024-04-24T22:40:23.915496+0000","last_peered":"2024-04-24T22:40:23.915496+0000","last_clean":"2024-04-24T22:40:23.915496+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:40:23.915496+0000","last_undegraded":"2024-04-24T22:40:23.915496+0000","last_fullsized":"2024-04-24T22:40:23.915496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.619238+0000","last_change":"2024-04-24T22:39:37.703786+0000","last_active":"2024-04-24T22:43:48.619238+0000","last_peered":"2024-04-24T22:43:48.619238+0000","last_clean":"2024-04-24T22:43:48.619238+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:43:48.619238+0000","last_undegraded":"2024-04-24T22:43:48.619238+0000","last_fullsized":"2024-04-24T22:43:48.619238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:55.990656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.907195+0000","last_change":"2024-04-24T22:39:36.715437+0000","last_active":"2024-04-24T22:40:23.907195+0000","last_peered":"2024-04-24T22:40:23.907195+0000","last_clean":"2024-04-24T22:40:23.907195+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:40:23.907195+0000","last_undegraded":"2024-04-24T22:40:23.907195+0000","last_fullsized":"2024-04-24T22:40:23.907195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.681549+0000","last_change":"2024-04-24T22:39:37.715801+0000","last_active":"2024-04-24T22:43:48.681549+0000","last_peered":"2024-04-24T22:43:48.681549+0000","last_clean":"2024-04-24T22:43:48.681549+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:43:48.681549+0000","last_undegraded":"2024-04-24T22:43:48.681549+0000","last_fullsized":"2024-04-24T22:43:48.681549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.915114+0000","last_change":"2024-04-24T22:39:36.710422+0000","last_active":"2024-04-24T22:40:23.915114+0000","last_peered":"2024-04-24T22:40:23.915114+0000","last_clean":"2024-04-24T22:40:23.915114+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:40:23.915114+0000","last_undegraded":"2024-04-24T22:40:23.915114+0000","last_fullsized":"2024-04-24T22:40:23.915114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'168","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.663298+0000","last_change":"2024-04-24T22:39:37.706745+0000","last_active":"2024-04-24T22:43:48.663298+0000","last_peered":"2024-04-24T22:43:48.663298+0000","last_clean":"2024-04-24T22:43:48.663298+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:43:48.663298+0000","last_undegraded":"2024-04-24T22:43:48.663298+0000","last_fullsized":"2024-04-24T22:43:48.663298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:34.691489+0000","last_change":"2024-04-24T22:39:32.657965+0000","last_active":"2024-04-24T22:41:34.691489+0000","last_peered":"2024-04-24T22:41:34.691489+0000","last_clean":"2024-04-24T22:41:34.691489+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:41:34.691489+0000","last_undegraded":"2024-04-24T22:41:34.691489+0000","last_fullsized":"2024-04-24T22:41:34.691489+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_clean_scrub_stamp":"2024-04-24T22:39:31.635804+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.714153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.694937+0000","last_change":"2024-04-24T22:39:36.710381+0000","last_active":"2024-04-24T22:41:44.694937+0000","last_peered":"2024-04-24T22:41:44.694937+0000","last_clean":"2024-04-24T22:41:44.694937+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:41:44.694937+0000","last_undegraded":"2024-04-24T22:41:44.694937+0000","last_fullsized":"2024-04-24T22:41:44.694937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.662348+0000","last_change":"2024-04-24T22:39:37.713956+0000","last_active":"2024-04-24T22:43:48.662348+0000","last_peered":"2024-04-24T22:43:48.662348+0000","last_clean":"2024-04-24T22:43:48.662348+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:43:48.662348+0000","last_undegraded":"2024-04-24T22:43:48.662348+0000","last_fullsized":"2024-04-24T22:43:48.662348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:39.691975+0000","last_change":"2024-04-24T22:39:36.715290+0000","last_active":"2024-04-24T22:41:39.691975+0000","last_peered":"2024-04-24T22:41:39.691975+0000","last_clean":"2024-04-24T22:41:39.691975+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:41:39.691975+0000","last_undegraded":"2024-04-24T22:41:39.691975+0000","last_fullsized":"2024-04-24T22:41:39.691975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.692616+0000","last_change":"2024-04-24T22:39:37.695419+0000","last_active":"2024-04-24T22:43:48.692616+0000","last_peered":"2024-04-24T22:43:48.692616+0000","last_clean":"2024-04-24T22:43:48.692616+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:43:48.692616+0000","last_undegraded":"2024-04-24T22:43:48.692616+0000","last_fullsized":"2024-04-24T22:43:48.692616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:08.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:58.090196+0000","last_change":"2024-04-24T22:39:36.707548+0000","last_active":"2024-04-24T22:43:58.090196+0000","last_peered":"2024-04-24T22:43:58.090196+0000","last_clean":"2024-04-24T22:43:58.090196+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:43:58.090196+0000","last_undegraded":"2024-04-24T22:43:58.090196+0000","last_fullsized":"2024-04-24T22:43:58.090196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673312+0000","last_change":"2024-04-24T22:39:37.703301+0000","last_active":"2024-04-24T22:43:48.673312+0000","last_peered":"2024-04-24T22:43:48.673312+0000","last_clean":"2024-04-24T22:43:48.673312+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:43:48.673312+0000","last_undegraded":"2024-04-24T22:43:48.673312+0000","last_fullsized":"2024-04-24T22:43:48.673312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.694926+0000","last_change":"2024-04-24T22:39:36.713486+0000","last_active":"2024-04-24T22:41:44.694926+0000","last_peered":"2024-04-24T22:41:44.694926+0000","last_clean":"2024-04-24T22:41:44.694926+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:41:44.694926+0000","last_undegraded":"2024-04-24T22:41:44.694926+0000","last_fullsized":"2024-04-24T22:41:44.694926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.694657+0000","last_change":"2024-04-24T22:39:37.705705+0000","last_active":"2024-04-24T22:43:48.694657+0000","last_peered":"2024-04-24T22:43:48.694657+0000","last_clean":"2024-04-24T22:43:48.694657+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:43:48.694657+0000","last_undegraded":"2024-04-24T22:43:48.694657+0000","last_fullsized":"2024-04-24T22:43:48.694657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:34.424877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.703126+0000","last_change":"2024-04-24T22:39:36.723125+0000","last_active":"2024-04-24T22:41:44.703126+0000","last_peered":"2024-04-24T22:41:44.703126+0000","last_clean":"2024-04-24T22:41:44.703126+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:41:44.703126+0000","last_undegraded":"2024-04-24T22:41:44.703126+0000","last_fullsized":"2024-04-24T22:41:44.703126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'155","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.676764+0000","last_change":"2024-04-24T22:39:37.713947+0000","last_active":"2024-04-24T22:43:48.676764+0000","last_peered":"2024-04-24T22:43:48.676764+0000","last_clean":"2024-04-24T22:43:48.676764+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:43:48.676764+0000","last_undegraded":"2024-04-24T22:43:48.676764+0000","last_fullsized":"2024-04-24T22:43:48.676764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.692421+0000","last_change":"2024-04-24T22:39:36.716541+0000","last_active":"2024-04-24T22:41:44.692421+0000","last_peered":"2024-04-24T22:41:44.692421+0000","last_clean":"2024-04-24T22:41:44.692421+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:41:44.692421+0000","last_undegraded":"2024-04-24T22:41:44.692421+0000","last_fullsized":"2024-04-24T22:41:44.692421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:33.366773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.694417+0000","last_change":"2024-04-24T22:39:37.708069+0000","last_active":"2024-04-24T22:43:48.694417+0000","last_peered":"2024-04-24T22:43:48.694417+0000","last_clean":"2024-04-24T22:43:48.694417+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:43:48.694417+0000","last_undegraded":"2024-04-24T22:43:48.694417+0000","last_fullsized":"2024-04-24T22:43:48.694417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'27","reported_seq":47,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:57.792124+0000","last_change":"2024-04-24T22:39:36.718740+0000","last_active":"2024-04-24T22:43:57.792124+0000","last_peered":"2024-04-24T22:43:57.792124+0000","last_clean":"2024-04-24T22:43:57.792124+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:43:57.792124+0000","last_undegraded":"2024-04-24T22:43:57.792124+0000","last_fullsized":"2024-04-24T22:43:57.792124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'161","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.679380+0000","last_change":"2024-04-24T22:39:37.704496+0000","last_active":"2024-04-24T22:43:48.679380+0000","last_peered":"2024-04-24T22:43:48.679380+0000","last_clean":"2024-04-24T22:43:48.679380+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:43:48.679380+0000","last_undegraded":"2024-04-24T22:43:48.679380+0000","last_fullsized":"2024-04-24T22:43:48.679380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:53.082764+0000","last_change":"2024-04-24T22:39:36.701308+0000","last_active":"2024-04-24T22:43:53.082764+0000","last_peered":"2024-04-24T22:43:53.082764+0000","last_clean":"2024-04-24T22:43:53.082764+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:43:53.082764+0000","last_undegraded":"2024-04-24T22:43:53.082764+0000","last_fullsized":"2024-04-24T22:43:53.082764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.683485+0000","last_change":"2024-04-24T22:39:37.700189+0000","last_active":"2024-04-24T22:43:48.683485+0000","last_peered":"2024-04-24T22:43:48.683485+0000","last_clean":"2024-04-24T22:43:48.683485+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:43:48.683485+0000","last_undegraded":"2024-04-24T22:43:48.683485+0000","last_fullsized":"2024-04-24T22:43:48.683485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:49.693021+0000","last_change":"2024-04-24T22:39:36.716460+0000","last_active":"2024-04-24T22:41:49.693021+0000","last_peered":"2024-04-24T22:41:49.693021+0000","last_clean":"2024-04-24T22:41:49.693021+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:41:49.693021+0000","last_undegraded":"2024-04-24T22:41:49.693021+0000","last_fullsized":"2024-04-24T22:41:49.693021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.693303+0000","last_change":"2024-04-24T22:39:37.704143+0000","last_active":"2024-04-24T22:43:48.693303+0000","last_peered":"2024-04-24T22:43:48.693303+0000","last_clean":"2024-04-24T22:43:48.693303+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:43:48.693303+0000","last_undegraded":"2024-04-24T22:43:48.693303+0000","last_fullsized":"2024-04-24T22:43:48.693303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:58.556519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:49.703968+0000","last_change":"2024-04-24T22:39:36.721540+0000","last_active":"2024-04-24T22:41:49.703968+0000","last_peered":"2024-04-24T22:41:49.703968+0000","last_clean":"2024-04-24T22:41:49.703968+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:41:49.703968+0000","last_undegraded":"2024-04-24T22:41:49.703968+0000","last_fullsized":"2024-04-24T22:41:49.703968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'164","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.664232+0000","last_change":"2024-04-24T22:39:37.714150+0000","last_active":"2024-04-24T22:43:48.664232+0000","last_peered":"2024-04-24T22:43:48.664232+0000","last_clean":"2024-04-24T22:43:48.664232+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:43:48.664232+0000","last_undegraded":"2024-04-24T22:43:48.664232+0000","last_fullsized":"2024-04-24T22:43:48.664232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:49.704015+0000","last_change":"2024-04-24T22:39:36.716667+0000","last_active":"2024-04-24T22:41:49.704015+0000","last_peered":"2024-04-24T22:41:49.704015+0000","last_clean":"2024-04-24T22:41:49.704015+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:41:49.704015+0000","last_undegraded":"2024-04-24T22:41:49.704015+0000","last_fullsized":"2024-04-24T22:41:49.704015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.675064+0000","last_change":"2024-04-24T22:39:37.715906+0000","last_active":"2024-04-24T22:43:48.675064+0000","last_peered":"2024-04-24T22:43:48.675064+0000","last_clean":"2024-04-24T22:43:48.675064+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:43:48.675064+0000","last_undegraded":"2024-04-24T22:43:48.675064+0000","last_fullsized":"2024-04-24T22:43:48.675064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:54.693418+0000","last_change":"2024-04-24T22:39:36.698212+0000","last_active":"2024-04-24T22:41:54.693418+0000","last_peered":"2024-04-24T22:41:54.693418+0000","last_clean":"2024-04-24T22:41:54.693418+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:41:54.693418+0000","last_undegraded":"2024-04-24T22:41:54.693418+0000","last_fullsized":"2024-04-24T22:41:54.693418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'149","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.686905+0000","last_change":"2024-04-24T22:39:37.696143+0000","last_active":"2024-04-24T22:43:48.686905+0000","last_peered":"2024-04-24T22:43:48.686905+0000","last_clean":"2024-04-24T22:43:48.686905+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:43:48.686905+0000","last_undegraded":"2024-04-24T22:43:48.686905+0000","last_fullsized":"2024-04-24T22:43:48.686905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'190","reported_seq":207,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.680917+0000","last_change":"2024-04-24T22:39:37.696113+0000","last_active":"2024-04-24T22:43:48.680917+0000","last_peered":"2024-04-24T22:43:48.680917+0000","last_clean":"2024-04-24T22:43:48.680917+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:43:48.680917+0000","last_undegraded":"2024-04-24T22:43:48.680917+0000","last_fullsized":"2024-04-24T22:43:48.680917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906632+0000","last_change":"2024-04-24T22:39:36.699149+0000","last_active":"2024-04-24T22:40:23.906632+0000","last_peered":"2024-04-24T22:40:23.906632+0000","last_clean":"2024-04-24T22:40:23.906632+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:40:23.906632+0000","last_undegraded":"2024-04-24T22:40:23.906632+0000","last_fullsized":"2024-04-24T22:40:23.906632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.665742+0000","last_change":"2024-04-24T22:39:37.700972+0000","last_active":"2024-04-24T22:43:48.665742+0000","last_peered":"2024-04-24T22:43:48.665742+0000","last_clean":"2024-04-24T22:43:48.665742+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:43:48.665742+0000","last_undegraded":"2024-04-24T22:43:48.665742+0000","last_fullsized":"2024-04-24T22:43:48.665742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'14","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:07.936964+0000","last_change":"2024-04-24T22:39:36.698567+0000","last_active":"2024-04-24T22:42:07.936964+0000","last_peered":"2024-04-24T22:42:07.936964+0000","last_clean":"2024-04-24T22:42:07.936964+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:42:07.936964+0000","last_undegraded":"2024-04-24T22:42:07.936964+0000","last_fullsized":"2024-04-24T22:42:07.936964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.661869+0000","last_change":"2024-04-24T22:39:37.706951+0000","last_active":"2024-04-24T22:43:48.661869+0000","last_peered":"2024-04-24T22:43:48.661869+0000","last_clean":"2024-04-24T22:43:48.661869+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:43:48.661869+0000","last_undegraded":"2024-04-24T22:43:48.661869+0000","last_fullsized":"2024-04-24T22:43:48.661869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:39.700252+0000","last_change":"2024-04-24T22:39:36.703823+0000","last_active":"2024-04-24T22:41:39.700252+0000","last_peered":"2024-04-24T22:41:39.700252+0000","last_clean":"2024-04-24T22:41:39.700252+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:41:39.700252+0000","last_undegraded":"2024-04-24T22:41:39.700252+0000","last_fullsized":"2024-04-24T22:41:39.700252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673604+0000","last_change":"2024-04-24T22:39:37.712078+0000","last_active":"2024-04-24T22:43:48.673604+0000","last_peered":"2024-04-24T22:43:48.673604+0000","last_clean":"2024-04-24T22:43:48.673604+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:43:48.673604+0000","last_undegraded":"2024-04-24T22:43:48.673604+0000","last_fullsized":"2024-04-24T22:43:48.673604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'13","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.706607+0000","last_change":"2024-04-24T22:39:36.719643+0000","last_active":"2024-04-24T22:41:44.706607+0000","last_peered":"2024-04-24T22:41:44.706607+0000","last_clean":"2024-04-24T22:41:44.706607+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:41:44.706607+0000","last_undegraded":"2024-04-24T22:41:44.706607+0000","last_fullsized":"2024-04-24T22:41:44.706607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":391015,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":413,"num_write_kb":583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032192,"data_stored":782030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":374,"ondisk_log_size":374,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40348,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331492,"statfs":{"total":96636764160,"available":96595447808,"internally_reserved":0,"allocated":872448,"data_stored":680263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":651264,"data_stored":472415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":290816,"data_stored":91985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39756,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332084,"statfs":{"total":96636764160,"available":96596054016,"internally_reserved":0,"allocated":266240,"data_stored":89125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":659456,"data_stored":472954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":278528,"data_stored":89762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":270336,"data_stored":89585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":864256,"data_stored":679037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":385159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":384620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:44:08.093 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:44:08.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.342+0000 7f996da00640 1 Processor -- start 2024-04-24T22:44:08.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.343+0000 7f996da00640 1 -- start start 2024-04-24T22:44:08.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.344+0000 7f996da00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f99681547a0 0x7f996815d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.344+0000 7f996da00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f996815b070 0x7f99680c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.344+0000 7f996da00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996815df50 con 0x7f996815ac50 2024-04-24T22:44:08.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.344+0000 7f996da00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f996815e230 con 0x7f99681547a0 2024-04-24T22:44:08.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.344+0000 7f996da00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f996815e510 con 0x7f996815b070 2024-04-24T22:44:08.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.345+0000 7f9967e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f99681547a0 0x7f996815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.345+0000 7f9967e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f99681547a0 0x7f996815d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60170/0 (socket says 172.21.15.47:60170) 2024-04-24T22:44:08.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.345+0000 7f9967e00640 1 -- 172.21.15.47:0/4180794265 learned_addr learned my addr 172.21.15.47:0/4180794265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:08.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.345+0000 7f996d000640 1 --2- 172.21.15.47:0/4180794265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f996815b070 0x7f99680c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.345+0000 7f9967e00640 1 -- 172.21.15.47:0/4180794265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f996815b070 msgr2=0x7f99680c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:08.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.345+0000 7f9967e00640 1 --2- 172.21.15.47:0/4180794265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f996815b070 0x7f99680c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.345+0000 7f9967e00640 1 -- 172.21.15.47:0/4180794265 >> v1:172.21.15.113:6789/0 conn(0x7f996815ac50 legacy=0x7f99680cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.345+0000 7f9967e00640 1 -- 172.21.15.47:0/4180794265 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996815e7f0 con 0x7f99681547a0 2024-04-24T22:44:08.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.345+0000 7f9966a00640 1 -- 172.21.15.47:0/4180794265 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3667347287 0 0) 0x7f996815df50 con 0x7f996815ac50 2024-04-24T22:44:08.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.346+0000 7f9967e00640 1 --2- 172.21.15.47:0/4180794265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99681547a0 0x7f996815d930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9958009870 tx=0x7f995802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8e811c2344aeb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:08.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.347+0000 7f9966a00640 1 -- 172.21.15.47:0/4180794265 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958037070 con 0x7f99681547a0 2024-04-24T22:44:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.347+0000 7f9966a00640 1 -- 172.21.15.47:0/4180794265 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9958032440 con 0x7f99681547a0 2024-04-24T22:44:08.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.347+0000 7f9966a00640 1 -- 172.21.15.47:0/4180794265 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958032810 con 0x7f99681547a0 2024-04-24T22:44:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.347+0000 7f996da00640 1 -- 172.21.15.47:0/4180794265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99681547a0 msgr2=0x7f996815d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.347+0000 7f996da00640 1 --2- 172.21.15.47:0/4180794265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99681547a0 0x7f996815d930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9958009870 tx=0x7f995802a250 comp rx=0 tx=0).stop 2024-04-24T22:44:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.348+0000 7f996da00640 1 -- 172.21.15.47:0/4180794265 shutdown_connections 2024-04-24T22:44:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.348+0000 7f996da00640 1 --2- 172.21.15.47:0/4180794265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f996815b070 0x7f99680c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.348+0000 7f996da00640 1 --2- 172.21.15.47:0/4180794265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99681547a0 0x7f996815d930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.348+0000 7f996da00640 1 -- 172.21.15.47:0/4180794265 >> 172.21.15.47:0/4180794265 conn(0x7f99680672c0 msgr2=0x7f9968072980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.348+0000 7f996da00640 1 -- 172.21.15.47:0/4180794265 shutdown_connections 2024-04-24T22:44:08.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.348+0000 7f996da00640 1 -- 172.21.15.47:0/4180794265 wait complete. 2024-04-24T22:44:08.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.349+0000 7f996da00640 1 Processor -- start 2024-04-24T22:44:08.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.350+0000 7f996da00640 1 -- start start 2024-04-24T22:44:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.350+0000 7f996da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99681547a0 0x7f99681f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.350+0000 7f996da00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f996815ac50 0x7f9968151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.350+0000 7f996da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f996815b070 0x7f9968151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.350+0000 7f996da00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f99680c3a90 con 0x7f99681547a0 2024-04-24T22:44:08.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.350+0000 7f996da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f996815a730 con 0x7f996815b070 2024-04-24T22:44:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.350+0000 7f996da00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f99680ce230 con 0x7f996815ac50 2024-04-24T22:44:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.351+0000 7f9967e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99681547a0 0x7f99681f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.351+0000 7f9967e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99681547a0 0x7f99681f0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35588/0 (socket says 172.21.15.47:35588) 2024-04-24T22:44:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.351+0000 7f9967e00640 1 -- 172.21.15.47:0/1318664149 learned_addr learned my addr 172.21.15.47:0/1318664149 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.351+0000 7f9967400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f996815ac50 0x7f9968151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.351+0000 7f996d000640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f996815b070 0x7f9968151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.351+0000 7f9967e00640 1 -- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f996815ac50 msgr2=0x7f9968151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.351+0000 7f9967e00640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f996815ac50 0x7f9968151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.351+0000 7f9967e00640 1 -- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f996815b070 msgr2=0x7f9968151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.351+0000 7f9967e00640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f996815b070 0x7f9968151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.351+0000 7f9967e00640 1 -- 172.21.15.47:0/1318664149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99681f11b0 con 0x7f99681547a0 2024-04-24T22:44:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.352+0000 7f9967e00640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99681547a0 0x7f99681f0c70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f99580090b0 tx=0x7f99580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.352+0000 7f995fe00640 1 -- 172.21.15.47:0/1318664149 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958037040 con 0x7f99681547a0 2024-04-24T22:44:08.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.352+0000 7f995fe00640 1 -- 172.21.15.47:0/1318664149 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9958004230 con 0x7f99681547a0 2024-04-24T22:44:08.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.352+0000 7f995fe00640 1 -- 172.21.15.47:0/1318664149 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958003610 con 0x7f99681547a0 2024-04-24T22:44:08.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.352+0000 7f996da00640 1 -- 172.21.15.47:0/1318664149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996815cf00 con 0x7f99681547a0 2024-04-24T22:44:08.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.352+0000 7f996da00640 1 -- 172.21.15.47:0/1318664149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99681f0470 con 0x7f99681547a0 2024-04-24T22:44:08.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.354+0000 7f996da00640 1 -- 172.21.15.47:0/1318664149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f996815d420 con 0x7f99681547a0 2024-04-24T22:44:08.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.354+0000 7f995fe00640 1 -- 172.21.15.47:0/1318664149 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99580037b0 con 0x7f99681547a0 2024-04-24T22:44:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.355+0000 7f995fe00640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f994406be60 0x7f994406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.355+0000 7f995fe00640 1 -- 172.21.15.47:0/1318664149 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f99580abaa0 con 0x7f99681547a0 2024-04-24T22:44:08.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.356+0000 7f9967400640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f994406be60 0x7f994406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.356+0000 7f9967400640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f994406be60 0x7f994406e320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9968152650 tx=0x7f9950009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:08.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.360+0000 7f995fe00640 1 -- 172.21.15.47:0/1318664149 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f996815d420 con 0x7f99681547a0 2024-04-24T22:44:08.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.559+0000 7f996da00640 1 -- 172.21.15.47:0/1318664149 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99681f0260 con 0x7f994406be60 2024-04-24T22:44:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.566+0000 7f995fe00640 1 -- 172.21.15.47:0/1318664149 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252947 (secure 0 0 0) 0x7f99681f0260 con 0x7f994406be60 2024-04-24T22:44:08.572 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:44:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 -- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f994406be60 msgr2=0x7f994406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f994406be60 0x7f994406e320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9968152650 tx=0x7f9950009040 comp rx=0 tx=0).stop 2024-04-24T22:44:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 -- 172.21.15.47:0/1318664149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99681547a0 msgr2=0x7f99681f0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99681547a0 0x7f99681f0c70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f99580090b0 tx=0x7f99580046c0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 -- 172.21.15.47:0/1318664149 shutdown_connections 2024-04-24T22:44:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f994406be60 0x7f994406e320 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f996815b070 0x7f9968151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f996815ac50 0x7f9968151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 --2- 172.21.15.47:0/1318664149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99681547a0 0x7f99681f0c70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 -- 172.21.15.47:0/1318664149 >> 172.21.15.47:0/1318664149 conn(0x7f99680672c0 msgr2=0x7f99680c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.571+0000 7f996da00640 1 -- 172.21.15.47:0/1318664149 shutdown_connections 2024-04-24T22:44:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.572+0000 7f996da00640 1 -- 172.21.15.47:0/1318664149 wait complete. 2024-04-24T22:44:08.595 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-24T22:44:06.703050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":981383,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":9788,"num_write_kb":25168823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11246,"ondisk_log_size":11246,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282584,"kb_used_data":4056,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754692136,"statfs":{"total":773094113280,"available":772804747264,"internally_reserved":0,"allocated":4153344,"data_stored":2665126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":285134387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9425,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003007"},"pg_stats":[{"pgid":"3.3f","version":"23'159","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.666893+0000","last_change":"2024-04-24T22:40:23.914828+0000","last_active":"2024-04-24T22:43:48.666893+0000","last_peered":"2024-04-24T22:43:48.666893+0000","last_clean":"2024-04-24T22:43:48.666893+0000","last_became_active":"2024-04-24T22:40:23.913848+0000","last_became_peered":"2024-04-24T22:40:23.913848+0000","last_unstale":"2024-04-24T22:43:48.666893+0000","last_undegraded":"2024-04-24T22:43:48.666893+0000","last_fullsized":"2024-04-24T22:43:48.666893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.690809+0000","last_change":"2024-04-24T22:39:37.697245+0000","last_active":"2024-04-24T22:43:48.690809+0000","last_peered":"2024-04-24T22:43:48.690809+0000","last_clean":"2024-04-24T22:43:48.690809+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:43:48.690809+0000","last_undegraded":"2024-04-24T22:43:48.690809+0000","last_fullsized":"2024-04-24T22:43:48.690809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.689091+0000","last_change":"2024-04-24T22:39:37.709539+0000","last_active":"2024-04-24T22:43:48.689091+0000","last_peered":"2024-04-24T22:43:48.689091+0000","last_clean":"2024-04-24T22:43:48.689091+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:43:48.689091+0000","last_undegraded":"2024-04-24T22:43:48.689091+0000","last_fullsized":"2024-04-24T22:43:48.689091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'185","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.691701+0000","last_change":"2024-04-24T22:39:37.719136+0000","last_active":"2024-04-24T22:43:48.691701+0000","last_peered":"2024-04-24T22:43:48.691701+0000","last_clean":"2024-04-24T22:43:48.691701+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:43:48.691701+0000","last_undegraded":"2024-04-24T22:43:48.691701+0000","last_fullsized":"2024-04-24T22:43:48.691701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.667187+0000","last_change":"2024-04-24T22:39:37.704671+0000","last_active":"2024-04-24T22:43:48.667187+0000","last_peered":"2024-04-24T22:43:48.667187+0000","last_clean":"2024-04-24T22:43:48.667187+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:43:48.667187+0000","last_undegraded":"2024-04-24T22:43:48.667187+0000","last_fullsized":"2024-04-24T22:43:48.667187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.683504+0000","last_change":"2024-04-24T22:39:37.703880+0000","last_active":"2024-04-24T22:43:48.683504+0000","last_peered":"2024-04-24T22:43:48.683504+0000","last_clean":"2024-04-24T22:43:48.683504+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:43:48.683504+0000","last_undegraded":"2024-04-24T22:43:48.683504+0000","last_fullsized":"2024-04-24T22:43:48.683504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.627153+0000","last_change":"2024-04-24T22:39:37.703539+0000","last_active":"2024-04-24T22:43:48.627153+0000","last_peered":"2024-04-24T22:43:48.627153+0000","last_clean":"2024-04-24T22:43:48.627153+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:43:48.627153+0000","last_undegraded":"2024-04-24T22:43:48.627153+0000","last_fullsized":"2024-04-24T22:43:48.627153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'195","reported_seq":212,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.682546+0000","last_change":"2024-04-24T22:39:37.713543+0000","last_active":"2024-04-24T22:43:48.682546+0000","last_peered":"2024-04-24T22:43:48.682546+0000","last_clean":"2024-04-24T22:43:48.682546+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:43:48.682546+0000","last_undegraded":"2024-04-24T22:43:48.682546+0000","last_fullsized":"2024-04-24T22:43:48.682546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'167","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.645645+0000","last_change":"2024-04-24T22:39:37.706715+0000","last_active":"2024-04-24T22:43:48.645645+0000","last_peered":"2024-04-24T22:43:48.645645+0000","last_clean":"2024-04-24T22:43:48.645645+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:43:48.645645+0000","last_undegraded":"2024-04-24T22:43:48.645645+0000","last_fullsized":"2024-04-24T22:43:48.645645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'175","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.672260+0000","last_change":"2024-04-24T22:40:23.915517+0000","last_active":"2024-04-24T22:43:48.672260+0000","last_peered":"2024-04-24T22:43:48.672260+0000","last_clean":"2024-04-24T22:43:48.672260+0000","last_became_active":"2024-04-24T22:40:23.915141+0000","last_became_peered":"2024-04-24T22:40:23.915141+0000","last_unstale":"2024-04-24T22:43:48.672260+0000","last_undegraded":"2024-04-24T22:43:48.672260+0000","last_fullsized":"2024-04-24T22:43:48.672260+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:07.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'208","reported_seq":225,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.677013+0000","last_change":"2024-04-24T22:39:37.701069+0000","last_active":"2024-04-24T22:43:48.677013+0000","last_peered":"2024-04-24T22:43:48.677013+0000","last_clean":"2024-04-24T22:43:48.677013+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:43:48.677013+0000","last_undegraded":"2024-04-24T22:43:48.677013+0000","last_fullsized":"2024-04-24T22:43:48.677013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.686235+0000","last_change":"2024-04-24T22:39:37.693508+0000","last_active":"2024-04-24T22:43:48.686235+0000","last_peered":"2024-04-24T22:43:48.686235+0000","last_clean":"2024-04-24T22:43:48.686235+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:43:48.686235+0000","last_undegraded":"2024-04-24T22:43:48.686235+0000","last_fullsized":"2024-04-24T22:43:48.686235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'167","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.669776+0000","last_change":"2024-04-24T22:39:37.705157+0000","last_active":"2024-04-24T22:43:48.669776+0000","last_peered":"2024-04-24T22:43:48.669776+0000","last_clean":"2024-04-24T22:43:48.669776+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:43:48.669776+0000","last_undegraded":"2024-04-24T22:43:48.669776+0000","last_fullsized":"2024-04-24T22:43:48.669776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'164","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.622338+0000","last_change":"2024-04-24T22:39:37.698333+0000","last_active":"2024-04-24T22:43:48.622338+0000","last_peered":"2024-04-24T22:43:48.622338+0000","last_clean":"2024-04-24T22:43:48.622338+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:43:48.622338+0000","last_undegraded":"2024-04-24T22:43:48.622338+0000","last_fullsized":"2024-04-24T22:43:48.622338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:57.510197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.671167+0000","last_change":"2024-04-24T22:39:37.696874+0000","last_active":"2024-04-24T22:43:48.671167+0000","last_peered":"2024-04-24T22:43:48.671167+0000","last_clean":"2024-04-24T22:43:48.671167+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:43:48.671167+0000","last_undegraded":"2024-04-24T22:43:48.671167+0000","last_fullsized":"2024-04-24T22:43:48.671167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.654424+0000","last_change":"2024-04-24T22:39:37.703446+0000","last_active":"2024-04-24T22:43:48.654424+0000","last_peered":"2024-04-24T22:43:48.654424+0000","last_clean":"2024-04-24T22:43:48.654424+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:43:48.654424+0000","last_undegraded":"2024-04-24T22:43:48.654424+0000","last_fullsized":"2024-04-24T22:43:48.654424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'168","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.666782+0000","last_change":"2024-04-24T22:40:23.916129+0000","last_active":"2024-04-24T22:43:48.666782+0000","last_peered":"2024-04-24T22:43:48.666782+0000","last_clean":"2024-04-24T22:43:48.666782+0000","last_became_active":"2024-04-24T22:40:23.915441+0000","last_became_peered":"2024-04-24T22:40:23.915441+0000","last_unstale":"2024-04-24T22:43:48.666782+0000","last_undegraded":"2024-04-24T22:43:48.666782+0000","last_fullsized":"2024-04-24T22:43:48.666782+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'169","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.672272+0000","last_change":"2024-04-24T22:39:37.703899+0000","last_active":"2024-04-24T22:43:48.672272+0000","last_peered":"2024-04-24T22:43:48.672272+0000","last_clean":"2024-04-24T22:43:48.672272+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:43:48.672272+0000","last_undegraded":"2024-04-24T22:43:48.672272+0000","last_fullsized":"2024-04-24T22:43:48.672272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'198","reported_seq":215,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.680661+0000","last_change":"2024-04-24T22:39:37.711019+0000","last_active":"2024-04-24T22:43:48.680661+0000","last_peered":"2024-04-24T22:43:48.680661+0000","last_clean":"2024-04-24T22:43:48.680661+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:43:48.680661+0000","last_undegraded":"2024-04-24T22:43:48.680661+0000","last_fullsized":"2024-04-24T22:43:48.680661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'171","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.685173+0000","last_change":"2024-04-24T22:39:37.696214+0000","last_active":"2024-04-24T22:43:48.685173+0000","last_peered":"2024-04-24T22:43:48.685173+0000","last_clean":"2024-04-24T22:43:48.685173+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:43:48.685173+0000","last_undegraded":"2024-04-24T22:43:48.685173+0000","last_fullsized":"2024-04-24T22:43:48.685173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'143","reported_seq":160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.654913+0000","last_change":"2024-04-24T22:39:37.700151+0000","last_active":"2024-04-24T22:43:48.654913+0000","last_peered":"2024-04-24T22:43:48.654913+0000","last_clean":"2024-04-24T22:43:48.654913+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:43:48.654913+0000","last_undegraded":"2024-04-24T22:43:48.654913+0000","last_fullsized":"2024-04-24T22:43:48.654913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'189","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.664027+0000","last_change":"2024-04-24T22:39:37.705083+0000","last_active":"2024-04-24T22:43:48.664027+0000","last_peered":"2024-04-24T22:43:48.664027+0000","last_clean":"2024-04-24T22:43:48.664027+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:43:48.664027+0000","last_undegraded":"2024-04-24T22:43:48.664027+0000","last_fullsized":"2024-04-24T22:43:48.664027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'142","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.652343+0000","last_change":"2024-04-24T22:39:37.711193+0000","last_active":"2024-04-24T22:43:48.652343+0000","last_peered":"2024-04-24T22:43:48.652343+0000","last_clean":"2024-04-24T22:43:48.652343+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:43:48.652343+0000","last_undegraded":"2024-04-24T22:43:48.652343+0000","last_fullsized":"2024-04-24T22:43:48.652343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'201","reported_seq":219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.685527+0000","last_change":"2024-04-24T22:39:37.713910+0000","last_active":"2024-04-24T22:43:48.685527+0000","last_peered":"2024-04-24T22:43:48.685527+0000","last_clean":"2024-04-24T22:43:48.685527+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:43:48.685527+0000","last_undegraded":"2024-04-24T22:43:48.685527+0000","last_fullsized":"2024-04-24T22:43:48.685527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'158","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.666804+0000","last_change":"2024-04-24T22:39:37.717191+0000","last_active":"2024-04-24T22:43:48.666804+0000","last_peered":"2024-04-24T22:43:48.666804+0000","last_clean":"2024-04-24T22:43:48.666804+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:43:48.666804+0000","last_undegraded":"2024-04-24T22:43:48.666804+0000","last_fullsized":"2024-04-24T22:43:48.666804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'202","reported_seq":220,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.638120+0000","last_change":"2024-04-24T22:39:37.702983+0000","last_active":"2024-04-24T22:43:48.638120+0000","last_peered":"2024-04-24T22:43:48.638120+0000","last_clean":"2024-04-24T22:43:48.638120+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:43:48.638120+0000","last_undegraded":"2024-04-24T22:43:48.638120+0000","last_fullsized":"2024-04-24T22:43:48.638120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.682048+0000","last_change":"2024-04-24T22:39:37.714111+0000","last_active":"2024-04-24T22:43:48.682048+0000","last_peered":"2024-04-24T22:43:48.682048+0000","last_clean":"2024-04-24T22:43:48.682048+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:43:48.682048+0000","last_undegraded":"2024-04-24T22:43:48.682048+0000","last_fullsized":"2024-04-24T22:43:48.682048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.669969+0000","last_change":"2024-04-24T22:39:37.714877+0000","last_active":"2024-04-24T22:43:48.669969+0000","last_peered":"2024-04-24T22:43:48.669969+0000","last_clean":"2024-04-24T22:43:48.669969+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:43:48.669969+0000","last_undegraded":"2024-04-24T22:43:48.669969+0000","last_fullsized":"2024-04-24T22:43:48.669969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'127","reported_seq":145,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.674841+0000","last_change":"2024-04-24T22:39:37.714695+0000","last_active":"2024-04-24T22:43:48.674841+0000","last_peered":"2024-04-24T22:43:48.674841+0000","last_clean":"2024-04-24T22:43:48.674841+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:43:48.674841+0000","last_undegraded":"2024-04-24T22:43:48.674841+0000","last_fullsized":"2024-04-24T22:43:48.674841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'145","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.656139+0000","last_change":"2024-04-24T22:39:37.719187+0000","last_active":"2024-04-24T22:43:48.656139+0000","last_peered":"2024-04-24T22:43:48.656139+0000","last_clean":"2024-04-24T22:43:48.656139+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:43:48.656139+0000","last_undegraded":"2024-04-24T22:43:48.656139+0000","last_fullsized":"2024-04-24T22:43:48.656139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'146","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.688744+0000","last_change":"2024-04-24T22:39:37.707283+0000","last_active":"2024-04-24T22:43:48.688744+0000","last_peered":"2024-04-24T22:43:48.688744+0000","last_clean":"2024-04-24T22:43:48.688744+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:43:48.688744+0000","last_undegraded":"2024-04-24T22:43:48.688744+0000","last_fullsized":"2024-04-24T22:43:48.688744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.683443+0000","last_change":"2024-04-24T22:39:37.697902+0000","last_active":"2024-04-24T22:43:48.683443+0000","last_peered":"2024-04-24T22:43:48.683443+0000","last_clean":"2024-04-24T22:43:48.683443+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:43:48.683443+0000","last_undegraded":"2024-04-24T22:43:48.683443+0000","last_fullsized":"2024-04-24T22:43:48.683443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'142","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.654469+0000","last_change":"2024-04-24T22:39:37.705667+0000","last_active":"2024-04-24T22:43:48.654469+0000","last_peered":"2024-04-24T22:43:48.654469+0000","last_clean":"2024-04-24T22:43:48.654469+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:43:48.654469+0000","last_undegraded":"2024-04-24T22:43:48.654469+0000","last_fullsized":"2024-04-24T22:43:48.654469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:50.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:24.451020+0000","last_change":"2024-04-24T22:39:36.707990+0000","last_active":"2024-04-24T22:40:24.451020+0000","last_peered":"2024-04-24T22:40:24.451020+0000","last_clean":"2024-04-24T22:40:24.451020+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:40:24.451020+0000","last_undegraded":"2024-04-24T22:40:24.451020+0000","last_fullsized":"2024-04-24T22:40:24.451020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'154","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.622757+0000","last_change":"2024-04-24T22:39:37.700406+0000","last_active":"2024-04-24T22:43:48.622757+0000","last_peered":"2024-04-24T22:43:48.622757+0000","last_clean":"2024-04-24T22:43:48.622757+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:43:48.622757+0000","last_undegraded":"2024-04-24T22:43:48.622757+0000","last_fullsized":"2024-04-24T22:43:48.622757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.908485+0000","last_change":"2024-04-24T22:39:36.701398+0000","last_active":"2024-04-24T22:40:23.908485+0000","last_peered":"2024-04-24T22:40:23.908485+0000","last_clean":"2024-04-24T22:40:23.908485+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:40:23.908485+0000","last_undegraded":"2024-04-24T22:40:23.908485+0000","last_fullsized":"2024-04-24T22:40:23.908485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:24.450888+0000","last_change":"2024-04-24T22:39:36.712294+0000","last_active":"2024-04-24T22:40:24.450888+0000","last_peered":"2024-04-24T22:40:24.450888+0000","last_clean":"2024-04-24T22:40:24.450888+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:40:24.450888+0000","last_undegraded":"2024-04-24T22:40:24.450888+0000","last_fullsized":"2024-04-24T22:40:24.450888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'182","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673441+0000","last_change":"2024-04-24T22:40:23.910148+0000","last_active":"2024-04-24T22:43:48.673441+0000","last_peered":"2024-04-24T22:43:48.673441+0000","last_clean":"2024-04-24T22:43:48.673441+0000","last_became_active":"2024-04-24T22:40:23.909573+0000","last_became_peered":"2024-04-24T22:40:23.909573+0000","last_unstale":"2024-04-24T22:43:48.673441+0000","last_undegraded":"2024-04-24T22:43:48.673441+0000","last_fullsized":"2024-04-24T22:43:48.673441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:07.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906152+0000","last_change":"2024-04-24T22:39:36.698983+0000","last_active":"2024-04-24T22:40:23.906152+0000","last_peered":"2024-04-24T22:40:23.906152+0000","last_clean":"2024-04-24T22:40:23.906152+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:40:23.906152+0000","last_undegraded":"2024-04-24T22:40:23.906152+0000","last_fullsized":"2024-04-24T22:40:23.906152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'177","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.676625+0000","last_change":"2024-04-24T22:39:37.714229+0000","last_active":"2024-04-24T22:43:48.676625+0000","last_peered":"2024-04-24T22:43:48.676625+0000","last_clean":"2024-04-24T22:43:48.676625+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:43:48.676625+0000","last_undegraded":"2024-04-24T22:43:48.676625+0000","last_fullsized":"2024-04-24T22:43:48.676625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:04.691585+0000","last_change":"2024-04-24T22:39:36.705776+0000","last_active":"2024-04-24T22:42:04.691585+0000","last_peered":"2024-04-24T22:42:04.691585+0000","last_clean":"2024-04-24T22:42:04.691585+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:42:04.691585+0000","last_undegraded":"2024-04-24T22:42:04.691585+0000","last_fullsized":"2024-04-24T22:42:04.691585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'241","reported_seq":258,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673855+0000","last_change":"2024-04-24T22:39:37.701436+0000","last_active":"2024-04-24T22:43:48.673855+0000","last_peered":"2024-04-24T22:43:48.673855+0000","last_clean":"2024-04-24T22:43:48.673855+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:43:48.673855+0000","last_undegraded":"2024-04-24T22:43:48.673855+0000","last_fullsized":"2024-04-24T22:43:48.673855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906331+0000","last_change":"2024-04-24T22:39:36.700237+0000","last_active":"2024-04-24T22:40:23.906331+0000","last_peered":"2024-04-24T22:40:23.906331+0000","last_clean":"2024-04-24T22:40:23.906331+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:40:23.906331+0000","last_undegraded":"2024-04-24T22:40:23.906331+0000","last_fullsized":"2024-04-24T22:40:23.906331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'151","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.660220+0000","last_change":"2024-04-24T22:39:37.703532+0000","last_active":"2024-04-24T22:43:48.660220+0000","last_peered":"2024-04-24T22:43:48.660220+0000","last_clean":"2024-04-24T22:43:48.660220+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:43:48.660220+0000","last_undegraded":"2024-04-24T22:43:48.660220+0000","last_fullsized":"2024-04-24T22:43:48.660220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'11","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:53.085629+0000","last_change":"2024-04-24T22:39:36.717477+0000","last_active":"2024-04-24T22:43:53.085629+0000","last_peered":"2024-04-24T22:43:53.085629+0000","last_clean":"2024-04-24T22:43:53.085629+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:43:53.085629+0000","last_undegraded":"2024-04-24T22:43:53.085629+0000","last_fullsized":"2024-04-24T22:43:53.085629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'157","reported_seq":174,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.640565+0000","last_change":"2024-04-24T22:39:37.699255+0000","last_active":"2024-04-24T22:43:48.640565+0000","last_peered":"2024-04-24T22:43:48.640565+0000","last_clean":"2024-04-24T22:43:48.640565+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:43:48.640565+0000","last_undegraded":"2024-04-24T22:43:48.640565+0000","last_fullsized":"2024-04-24T22:43:48.640565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'166","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:53.750307+0000","last_change":"2024-04-24T22:39:36.706567+0000","last_active":"2024-04-24T22:43:53.750307+0000","last_peered":"2024-04-24T22:43:53.750307+0000","last_clean":"2024-04-24T22:43:53.750307+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:43:53.750307+0000","last_undegraded":"2024-04-24T22:43:53.750307+0000","last_fullsized":"2024-04-24T22:43:53.750307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383830,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.670888+0000","last_change":"2024-04-24T22:39:37.712064+0000","last_active":"2024-04-24T22:43:48.670888+0000","last_peered":"2024-04-24T22:43:48.670888+0000","last_clean":"2024-04-24T22:43:48.670888+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:43:48.670888+0000","last_undegraded":"2024-04-24T22:43:48.670888+0000","last_fullsized":"2024-04-24T22:43:48.670888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'9","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.903984+0000","last_change":"2024-04-24T22:39:36.700041+0000","last_active":"2024-04-24T22:40:23.903984+0000","last_peered":"2024-04-24T22:40:23.903984+0000","last_clean":"2024-04-24T22:40:23.903984+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:40:23.903984+0000","last_undegraded":"2024-04-24T22:40:23.903984+0000","last_fullsized":"2024-04-24T22:40:23.903984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:50.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'168","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.624474+0000","last_change":"2024-04-24T22:39:37.691459+0000","last_active":"2024-04-24T22:43:48.624474+0000","last_peered":"2024-04-24T22:43:48.624474+0000","last_clean":"2024-04-24T22:43:48.624474+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:43:48.624474+0000","last_undegraded":"2024-04-24T22:43:48.624474+0000","last_fullsized":"2024-04-24T22:43:48.624474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906390+0000","last_change":"2024-04-24T22:39:36.706454+0000","last_active":"2024-04-24T22:40:23.906390+0000","last_peered":"2024-04-24T22:40:23.906390+0000","last_clean":"2024-04-24T22:40:23.906390+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:40:23.906390+0000","last_undegraded":"2024-04-24T22:40:23.906390+0000","last_fullsized":"2024-04-24T22:40:23.906390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'186","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.675788+0000","last_change":"2024-04-24T22:39:37.696724+0000","last_active":"2024-04-24T22:43:48.675788+0000","last_peered":"2024-04-24T22:43:48.675788+0000","last_clean":"2024-04-24T22:43:48.675788+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:43:48.675788+0000","last_undegraded":"2024-04-24T22:43:48.675788+0000","last_fullsized":"2024-04-24T22:43:48.675788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.908547+0000","last_change":"2024-04-24T22:39:36.701340+0000","last_active":"2024-04-24T22:40:23.908547+0000","last_peered":"2024-04-24T22:40:23.908547+0000","last_clean":"2024-04-24T22:40:23.908547+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:40:23.908547+0000","last_undegraded":"2024-04-24T22:40:23.908547+0000","last_fullsized":"2024-04-24T22:40:23.908547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'191","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.679977+0000","last_change":"2024-04-24T22:39:37.697532+0000","last_active":"2024-04-24T22:43:48.679977+0000","last_peered":"2024-04-24T22:43:48.679977+0000","last_clean":"2024-04-24T22:43:48.679977+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:43:48.679977+0000","last_undegraded":"2024-04-24T22:43:48.679977+0000","last_fullsized":"2024-04-24T22:43:48.679977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'18","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:59.706381+0000","last_change":"2024-04-24T22:39:36.724649+0000","last_active":"2024-04-24T22:41:59.706381+0000","last_peered":"2024-04-24T22:41:59.706381+0000","last_clean":"2024-04-24T22:41:59.706381+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:41:59.706381+0000","last_undegraded":"2024-04-24T22:41:59.706381+0000","last_fullsized":"2024-04-24T22:41:59.706381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'209","reported_seq":226,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.637954+0000","last_change":"2024-04-24T22:39:37.702890+0000","last_active":"2024-04-24T22:43:48.637954+0000","last_peered":"2024-04-24T22:43:48.637954+0000","last_clean":"2024-04-24T22:43:48.637954+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:43:48.637954+0000","last_undegraded":"2024-04-24T22:43:48.637954+0000","last_fullsized":"2024-04-24T22:43:48.637954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:24.792909+0000","last_change":"2024-04-24T22:39:36.716044+0000","last_active":"2024-04-24T22:40:24.792909+0000","last_peered":"2024-04-24T22:40:24.792909+0000","last_clean":"2024-04-24T22:40:24.792909+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:40:24.792909+0000","last_undegraded":"2024-04-24T22:40:24.792909+0000","last_fullsized":"2024-04-24T22:40:24.792909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:40.023933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.692442+0000","last_change":"2024-04-24T22:39:37.702621+0000","last_active":"2024-04-24T22:43:48.692442+0000","last_peered":"2024-04-24T22:43:48.692442+0000","last_clean":"2024-04-24T22:43:48.692442+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:43:48.692442+0000","last_undegraded":"2024-04-24T22:43:48.692442+0000","last_fullsized":"2024-04-24T22:43:48.692442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.915496+0000","last_change":"2024-04-24T22:39:36.714272+0000","last_active":"2024-04-24T22:40:23.915496+0000","last_peered":"2024-04-24T22:40:23.915496+0000","last_clean":"2024-04-24T22:40:23.915496+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:40:23.915496+0000","last_undegraded":"2024-04-24T22:40:23.915496+0000","last_fullsized":"2024-04-24T22:40:23.915496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.619238+0000","last_change":"2024-04-24T22:39:37.703786+0000","last_active":"2024-04-24T22:43:48.619238+0000","last_peered":"2024-04-24T22:43:48.619238+0000","last_clean":"2024-04-24T22:43:48.619238+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:43:48.619238+0000","last_undegraded":"2024-04-24T22:43:48.619238+0000","last_fullsized":"2024-04-24T22:43:48.619238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:55.990656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.907195+0000","last_change":"2024-04-24T22:39:36.715437+0000","last_active":"2024-04-24T22:40:23.907195+0000","last_peered":"2024-04-24T22:40:23.907195+0000","last_clean":"2024-04-24T22:40:23.907195+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:40:23.907195+0000","last_undegraded":"2024-04-24T22:40:23.907195+0000","last_fullsized":"2024-04-24T22:40:23.907195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.681549+0000","last_change":"2024-04-24T22:39:37.715801+0000","last_active":"2024-04-24T22:43:48.681549+0000","last_peered":"2024-04-24T22:43:48.681549+0000","last_clean":"2024-04-24T22:43:48.681549+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:43:48.681549+0000","last_undegraded":"2024-04-24T22:43:48.681549+0000","last_fullsized":"2024-04-24T22:43:48.681549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.915114+0000","last_change":"2024-04-24T22:39:36.710422+0000","last_active":"2024-04-24T22:40:23.915114+0000","last_peered":"2024-04-24T22:40:23.915114+0000","last_clean":"2024-04-24T22:40:23.915114+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:40:23.915114+0000","last_undegraded":"2024-04-24T22:40:23.915114+0000","last_fullsized":"2024-04-24T22:40:23.915114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'168","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.663298+0000","last_change":"2024-04-24T22:39:37.706745+0000","last_active":"2024-04-24T22:43:48.663298+0000","last_peered":"2024-04-24T22:43:48.663298+0000","last_clean":"2024-04-24T22:43:48.663298+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:43:48.663298+0000","last_undegraded":"2024-04-24T22:43:48.663298+0000","last_fullsized":"2024-04-24T22:43:48.663298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:34.691489+0000","last_change":"2024-04-24T22:39:32.657965+0000","last_active":"2024-04-24T22:41:34.691489+0000","last_peered":"2024-04-24T22:41:34.691489+0000","last_clean":"2024-04-24T22:41:34.691489+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:41:34.691489+0000","last_undegraded":"2024-04-24T22:41:34.691489+0000","last_fullsized":"2024-04-24T22:41:34.691489+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_clean_scrub_stamp":"2024-04-24T22:39:31.635804+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.714153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.694937+0000","last_change":"2024-04-24T22:39:36.710381+0000","last_active":"2024-04-24T22:41:44.694937+0000","last_peered":"2024-04-24T22:41:44.694937+0000","last_clean":"2024-04-24T22:41:44.694937+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:41:44.694937+0000","last_undegraded":"2024-04-24T22:41:44.694937+0000","last_fullsized":"2024-04-24T22:41:44.694937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.662348+0000","last_change":"2024-04-24T22:39:37.713956+0000","last_active":"2024-04-24T22:43:48.662348+0000","last_peered":"2024-04-24T22:43:48.662348+0000","last_clean":"2024-04-24T22:43:48.662348+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:43:48.662348+0000","last_undegraded":"2024-04-24T22:43:48.662348+0000","last_fullsized":"2024-04-24T22:43:48.662348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:39.691975+0000","last_change":"2024-04-24T22:39:36.715290+0000","last_active":"2024-04-24T22:41:39.691975+0000","last_peered":"2024-04-24T22:41:39.691975+0000","last_clean":"2024-04-24T22:41:39.691975+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:41:39.691975+0000","last_undegraded":"2024-04-24T22:41:39.691975+0000","last_fullsized":"2024-04-24T22:41:39.691975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.692616+0000","last_change":"2024-04-24T22:39:37.695419+0000","last_active":"2024-04-24T22:43:48.692616+0000","last_peered":"2024-04-24T22:43:48.692616+0000","last_clean":"2024-04-24T22:43:48.692616+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:43:48.692616+0000","last_undegraded":"2024-04-24T22:43:48.692616+0000","last_fullsized":"2024-04-24T22:43:48.692616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:08.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:58.090196+0000","last_change":"2024-04-24T22:39:36.707548+0000","last_active":"2024-04-24T22:43:58.090196+0000","last_peered":"2024-04-24T22:43:58.090196+0000","last_clean":"2024-04-24T22:43:58.090196+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:43:58.090196+0000","last_undegraded":"2024-04-24T22:43:58.090196+0000","last_fullsized":"2024-04-24T22:43:58.090196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673312+0000","last_change":"2024-04-24T22:39:37.703301+0000","last_active":"2024-04-24T22:43:48.673312+0000","last_peered":"2024-04-24T22:43:48.673312+0000","last_clean":"2024-04-24T22:43:48.673312+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:43:48.673312+0000","last_undegraded":"2024-04-24T22:43:48.673312+0000","last_fullsized":"2024-04-24T22:43:48.673312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.694926+0000","last_change":"2024-04-24T22:39:36.713486+0000","last_active":"2024-04-24T22:41:44.694926+0000","last_peered":"2024-04-24T22:41:44.694926+0000","last_clean":"2024-04-24T22:41:44.694926+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:41:44.694926+0000","last_undegraded":"2024-04-24T22:41:44.694926+0000","last_fullsized":"2024-04-24T22:41:44.694926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.694657+0000","last_change":"2024-04-24T22:39:37.705705+0000","last_active":"2024-04-24T22:43:48.694657+0000","last_peered":"2024-04-24T22:43:48.694657+0000","last_clean":"2024-04-24T22:43:48.694657+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:43:48.694657+0000","last_undegraded":"2024-04-24T22:43:48.694657+0000","last_fullsized":"2024-04-24T22:43:48.694657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:34.424877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.703126+0000","last_change":"2024-04-24T22:39:36.723125+0000","last_active":"2024-04-24T22:41:44.703126+0000","last_peered":"2024-04-24T22:41:44.703126+0000","last_clean":"2024-04-24T22:41:44.703126+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:41:44.703126+0000","last_undegraded":"2024-04-24T22:41:44.703126+0000","last_fullsized":"2024-04-24T22:41:44.703126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'155","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.676764+0000","last_change":"2024-04-24T22:39:37.713947+0000","last_active":"2024-04-24T22:43:48.676764+0000","last_peered":"2024-04-24T22:43:48.676764+0000","last_clean":"2024-04-24T22:43:48.676764+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:43:48.676764+0000","last_undegraded":"2024-04-24T22:43:48.676764+0000","last_fullsized":"2024-04-24T22:43:48.676764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.692421+0000","last_change":"2024-04-24T22:39:36.716541+0000","last_active":"2024-04-24T22:41:44.692421+0000","last_peered":"2024-04-24T22:41:44.692421+0000","last_clean":"2024-04-24T22:41:44.692421+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:41:44.692421+0000","last_undegraded":"2024-04-24T22:41:44.692421+0000","last_fullsized":"2024-04-24T22:41:44.692421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:33.366773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.694417+0000","last_change":"2024-04-24T22:39:37.708069+0000","last_active":"2024-04-24T22:43:48.694417+0000","last_peered":"2024-04-24T22:43:48.694417+0000","last_clean":"2024-04-24T22:43:48.694417+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:43:48.694417+0000","last_undegraded":"2024-04-24T22:43:48.694417+0000","last_fullsized":"2024-04-24T22:43:48.694417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'27","reported_seq":47,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:57.792124+0000","last_change":"2024-04-24T22:39:36.718740+0000","last_active":"2024-04-24T22:43:57.792124+0000","last_peered":"2024-04-24T22:43:57.792124+0000","last_clean":"2024-04-24T22:43:57.792124+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:43:57.792124+0000","last_undegraded":"2024-04-24T22:43:57.792124+0000","last_fullsized":"2024-04-24T22:43:57.792124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'161","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.679380+0000","last_change":"2024-04-24T22:39:37.704496+0000","last_active":"2024-04-24T22:43:48.679380+0000","last_peered":"2024-04-24T22:43:48.679380+0000","last_clean":"2024-04-24T22:43:48.679380+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:43:48.679380+0000","last_undegraded":"2024-04-24T22:43:48.679380+0000","last_fullsized":"2024-04-24T22:43:48.679380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:53.082764+0000","last_change":"2024-04-24T22:39:36.701308+0000","last_active":"2024-04-24T22:43:53.082764+0000","last_peered":"2024-04-24T22:43:53.082764+0000","last_clean":"2024-04-24T22:43:53.082764+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:43:53.082764+0000","last_undegraded":"2024-04-24T22:43:53.082764+0000","last_fullsized":"2024-04-24T22:43:53.082764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.683485+0000","last_change":"2024-04-24T22:39:37.700189+0000","last_active":"2024-04-24T22:43:48.683485+0000","last_peered":"2024-04-24T22:43:48.683485+0000","last_clean":"2024-04-24T22:43:48.683485+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:43:48.683485+0000","last_undegraded":"2024-04-24T22:43:48.683485+0000","last_fullsized":"2024-04-24T22:43:48.683485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:49.693021+0000","last_change":"2024-04-24T22:39:36.716460+0000","last_active":"2024-04-24T22:41:49.693021+0000","last_peered":"2024-04-24T22:41:49.693021+0000","last_clean":"2024-04-24T22:41:49.693021+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:41:49.693021+0000","last_undegraded":"2024-04-24T22:41:49.693021+0000","last_fullsized":"2024-04-24T22:41:49.693021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.693303+0000","last_change":"2024-04-24T22:39:37.704143+0000","last_active":"2024-04-24T22:43:48.693303+0000","last_peered":"2024-04-24T22:43:48.693303+0000","last_clean":"2024-04-24T22:43:48.693303+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:43:48.693303+0000","last_undegraded":"2024-04-24T22:43:48.693303+0000","last_fullsized":"2024-04-24T22:43:48.693303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:58.556519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:49.703968+0000","last_change":"2024-04-24T22:39:36.721540+0000","last_active":"2024-04-24T22:41:49.703968+0000","last_peered":"2024-04-24T22:41:49.703968+0000","last_clean":"2024-04-24T22:41:49.703968+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:41:49.703968+0000","last_undegraded":"2024-04-24T22:41:49.703968+0000","last_fullsized":"2024-04-24T22:41:49.703968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'164","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.664232+0000","last_change":"2024-04-24T22:39:37.714150+0000","last_active":"2024-04-24T22:43:48.664232+0000","last_peered":"2024-04-24T22:43:48.664232+0000","last_clean":"2024-04-24T22:43:48.664232+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:43:48.664232+0000","last_undegraded":"2024-04-24T22:43:48.664232+0000","last_fullsized":"2024-04-24T22:43:48.664232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:49.704015+0000","last_change":"2024-04-24T22:39:36.716667+0000","last_active":"2024-04-24T22:41:49.704015+0000","last_peered":"2024-04-24T22:41:49.704015+0000","last_clean":"2024-04-24T22:41:49.704015+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:41:49.704015+0000","last_undegraded":"2024-04-24T22:41:49.704015+0000","last_fullsized":"2024-04-24T22:41:49.704015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.675064+0000","last_change":"2024-04-24T22:39:37.715906+0000","last_active":"2024-04-24T22:43:48.675064+0000","last_peered":"2024-04-24T22:43:48.675064+0000","last_clean":"2024-04-24T22:43:48.675064+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:43:48.675064+0000","last_undegraded":"2024-04-24T22:43:48.675064+0000","last_fullsized":"2024-04-24T22:43:48.675064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:54.693418+0000","last_change":"2024-04-24T22:39:36.698212+0000","last_active":"2024-04-24T22:41:54.693418+0000","last_peered":"2024-04-24T22:41:54.693418+0000","last_clean":"2024-04-24T22:41:54.693418+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:41:54.693418+0000","last_undegraded":"2024-04-24T22:41:54.693418+0000","last_fullsized":"2024-04-24T22:41:54.693418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'149","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.686905+0000","last_change":"2024-04-24T22:39:37.696143+0000","last_active":"2024-04-24T22:43:48.686905+0000","last_peered":"2024-04-24T22:43:48.686905+0000","last_clean":"2024-04-24T22:43:48.686905+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:43:48.686905+0000","last_undegraded":"2024-04-24T22:43:48.686905+0000","last_fullsized":"2024-04-24T22:43:48.686905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'190","reported_seq":207,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.680917+0000","last_change":"2024-04-24T22:39:37.696113+0000","last_active":"2024-04-24T22:43:48.680917+0000","last_peered":"2024-04-24T22:43:48.680917+0000","last_clean":"2024-04-24T22:43:48.680917+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:43:48.680917+0000","last_undegraded":"2024-04-24T22:43:48.680917+0000","last_fullsized":"2024-04-24T22:43:48.680917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906632+0000","last_change":"2024-04-24T22:39:36.699149+0000","last_active":"2024-04-24T22:40:23.906632+0000","last_peered":"2024-04-24T22:40:23.906632+0000","last_clean":"2024-04-24T22:40:23.906632+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:40:23.906632+0000","last_undegraded":"2024-04-24T22:40:23.906632+0000","last_fullsized":"2024-04-24T22:40:23.906632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.665742+0000","last_change":"2024-04-24T22:39:37.700972+0000","last_active":"2024-04-24T22:43:48.665742+0000","last_peered":"2024-04-24T22:43:48.665742+0000","last_clean":"2024-04-24T22:43:48.665742+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:43:48.665742+0000","last_undegraded":"2024-04-24T22:43:48.665742+0000","last_fullsized":"2024-04-24T22:43:48.665742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'14","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:07.936964+0000","last_change":"2024-04-24T22:39:36.698567+0000","last_active":"2024-04-24T22:42:07.936964+0000","last_peered":"2024-04-24T22:42:07.936964+0000","last_clean":"2024-04-24T22:42:07.936964+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:42:07.936964+0000","last_undegraded":"2024-04-24T22:42:07.936964+0000","last_fullsized":"2024-04-24T22:42:07.936964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.661869+0000","last_change":"2024-04-24T22:39:37.706951+0000","last_active":"2024-04-24T22:43:48.661869+0000","last_peered":"2024-04-24T22:43:48.661869+0000","last_clean":"2024-04-24T22:43:48.661869+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:43:48.661869+0000","last_undegraded":"2024-04-24T22:43:48.661869+0000","last_fullsized":"2024-04-24T22:43:48.661869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:39.700252+0000","last_change":"2024-04-24T22:39:36.703823+0000","last_active":"2024-04-24T22:41:39.700252+0000","last_peered":"2024-04-24T22:41:39.700252+0000","last_clean":"2024-04-24T22:41:39.700252+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:41:39.700252+0000","last_undegraded":"2024-04-24T22:41:39.700252+0000","last_fullsized":"2024-04-24T22:41:39.700252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673604+0000","last_change":"2024-04-24T22:39:37.712078+0000","last_active":"2024-04-24T22:43:48.673604+0000","last_peered":"2024-04-24T22:43:48.673604+0000","last_clean":"2024-04-24T22:43:48.673604+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:43:48.673604+0000","last_undegraded":"2024-04-24T22:43:48.673604+0000","last_fullsized":"2024-04-24T22:43:48.673604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'13","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.706607+0000","last_change":"2024-04-24T22:39:36.719643+0000","last_active":"2024-04-24T22:41:44.706607+0000","last_peered":"2024-04-24T22:41:44.706607+0000","last_clean":"2024-04-24T22:41:44.706607+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:41:44.706607+0000","last_undegraded":"2024-04-24T22:41:44.706607+0000","last_fullsized":"2024-04-24T22:41:44.706607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":391015,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":413,"num_write_kb":583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032192,"data_stored":782030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":374,"ondisk_log_size":374,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40348,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331492,"statfs":{"total":96636764160,"available":96595447808,"internally_reserved":0,"allocated":872448,"data_stored":680263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":651264,"data_stored":472415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":290816,"data_stored":91985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39756,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332084,"statfs":{"total":96636764160,"available":96596054016,"internally_reserved":0,"allocated":266240,"data_stored":89125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":659456,"data_stored":472954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":278528,"data_stored":89762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":270336,"data_stored":89585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":864256,"data_stored":679037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":385159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":384620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:44:08.599 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T22:44:08.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:44:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.852+0000 7fa070000640 1 Processor -- start 2024-04-24T22:44:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.852+0000 7fa070000640 1 -- start start 2024-04-24T22:44:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.854+0000 7fa070000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa068154960 0x7fa06815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.854+0000 7fa070000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa06815e830 0x7fa0680c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.854+0000 7fa070000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa06815ab10 con 0x7fa06815a680 2024-04-24T22:44:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.854+0000 7fa070000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa06815adf0 con 0x7fa06815e830 2024-04-24T22:44:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.854+0000 7fa070000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa06815b0d0 con 0x7fa068154960 2024-04-24T22:44:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.854+0000 7fa06e000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa06815e830 0x7fa0680c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.854+0000 7fa06e000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa06815e830 0x7fa0680c1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35590/0 (socket says 172.21.15.47:35590) 2024-04-24T22:44:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.854+0000 7fa06e000640 1 -- 172.21.15.47:0/2857928885 learned_addr learned my addr 172.21.15.47:0/2857928885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.854+0000 7fa06d600640 1 --2- 172.21.15.47:0/2857928885 >> v2:172.21.15.113:3300/0 conn(0x7fa068154960 0x7fa06815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.855+0000 7fa06e000640 1 -- 172.21.15.47:0/2857928885 >> v2:172.21.15.113:3300/0 conn(0x7fa068154960 msgr2=0x7fa06815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.855+0000 7fa06e000640 1 --2- 172.21.15.47:0/2857928885 >> v2:172.21.15.113:3300/0 conn(0x7fa068154960 0x7fa06815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.855+0000 7fa06e000640 1 -- 172.21.15.47:0/2857928885 >> v1:172.21.15.47:6789/0 conn(0x7fa06815a680 legacy=0x7fa0680c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.855+0000 7fa06e000640 1 -- 172.21.15.47:0/2857928885 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06815b3b0 con 0x7fa06815e830 2024-04-24T22:44:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.856+0000 7fa06e000640 1 --2- 172.21.15.47:0/2857928885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa06815e830 0x7fa0680c1070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa06806f540 tx=0x7fa06002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49bc8774c9f8d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.856+0000 7fa067e00640 1 -- 172.21.15.47:0/2857928885 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060037070 con 0x7fa06815e830 2024-04-24T22:44:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.856+0000 7fa067e00640 1 -- 172.21.15.47:0/2857928885 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa060032440 con 0x7fa06815e830 2024-04-24T22:44:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.856+0000 7fa067e00640 1 -- 172.21.15.47:0/2857928885 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060032810 con 0x7fa06815e830 2024-04-24T22:44:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.857+0000 7fa070000640 1 -- 172.21.15.47:0/2857928885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa06815e830 msgr2=0x7fa0680c1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.857+0000 7fa070000640 1 --2- 172.21.15.47:0/2857928885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa06815e830 0x7fa0680c1070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa06806f540 tx=0x7fa06002a270 comp rx=0 tx=0).stop 2024-04-24T22:44:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.857+0000 7fa070000640 1 -- 172.21.15.47:0/2857928885 shutdown_connections 2024-04-24T22:44:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.857+0000 7fa070000640 1 --2- 172.21.15.47:0/2857928885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa06815e830 0x7fa0680c1070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.857+0000 7fa070000640 1 --2- 172.21.15.47:0/2857928885 >> v2:172.21.15.113:3300/0 conn(0x7fa068154960 0x7fa06815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.857+0000 7fa070000640 1 -- 172.21.15.47:0/2857928885 >> 172.21.15.47:0/2857928885 conn(0x7fa06806e580 msgr2=0x7fa06806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.857+0000 7fa070000640 1 -- 172.21.15.47:0/2857928885 shutdown_connections 2024-04-24T22:44:08.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.857+0000 7fa070000640 1 -- 172.21.15.47:0/2857928885 wait complete. 2024-04-24T22:44:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.858+0000 7fa070000640 1 Processor -- start 2024-04-24T22:44:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.859+0000 7fa070000640 1 -- start start 2024-04-24T22:44:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.859+0000 7fa070000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068154960 0x7fa068153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa070000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa06815e830 0x7fa068153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa070000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa06814d580 0x7fa06814da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa070000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa0680c8ca0 con 0x7fa068154960 2024-04-24T22:44:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa070000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa068154d40 con 0x7fa06814d580 2024-04-24T22:44:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa070000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa0680c8e20 con 0x7fa06815e830 2024-04-24T22:44:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa06d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068154960 0x7fa068153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa06e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa06814d580 0x7fa06814da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa06d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068154960 0x7fa068153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35602/0 (socket says 172.21.15.47:35602) 2024-04-24T22:44:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa06d600640 1 -- 172.21.15.47:0/3589905067 learned_addr learned my addr 172.21.15.47:0/3589905067 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa06cc00640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa06815e830 0x7fa068153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa06e000640 1 -- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa06815e830 msgr2=0x7fa068153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa06e000640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa06815e830 0x7fa068153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa06e000640 1 -- 172.21.15.47:0/3589905067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068154960 msgr2=0x7fa068153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa06e000640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068154960 0x7fa068153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.860+0000 7fa06e000640 1 -- 172.21.15.47:0/3589905067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0681523c0 con 0x7fa06814d580 2024-04-24T22:44:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.861+0000 7fa06e000640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa06814d580 0x7fa06814da10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa060031000 tx=0x7fa0600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.861+0000 7fa065600640 1 -- 172.21.15.47:0/3589905067 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060041020 con 0x7fa06814d580 2024-04-24T22:44:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.861+0000 7fa065600640 1 -- 172.21.15.47:0/3589905067 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa060030660 con 0x7fa06814d580 2024-04-24T22:44:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.861+0000 7fa070000640 1 -- 172.21.15.47:0/3589905067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06815cc60 con 0x7fa06814d580 2024-04-24T22:44:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.861+0000 7fa065600640 1 -- 172.21.15.47:0/3589905067 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060037040 con 0x7fa06814d580 2024-04-24T22:44:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.861+0000 7fa070000640 1 -- 172.21.15.47:0/3589905067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa068152660 con 0x7fa06814d580 2024-04-24T22:44:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.862+0000 7fa070000640 1 -- 172.21.15.47:0/3589905067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0680c61f0 con 0x7fa06814d580 2024-04-24T22:44:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.863+0000 7fa065600640 1 -- 172.21.15.47:0/3589905067 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa060044020 con 0x7fa06814d580 2024-04-24T22:44:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.864+0000 7fa065600640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa04806be30 0x7fa04806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.864+0000 7fa065600640 1 -- 172.21.15.47:0/3589905067 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa0600ab4e0 con 0x7fa06814d580 2024-04-24T22:44:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.864+0000 7fa06d600640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa04806be30 0x7fa04806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.865+0000 7fa06d600640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa04806be30 0x7fa04806e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009a40 tx=0x7fa05c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:08.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:08.868+0000 7fa065600640 1 -- 172.21.15.47:0/3589905067 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa0680c61f0 con 0x7fa06814d580 2024-04-24T22:44:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.068+0000 7fa070000640 1 -- 172.21.15.47:0/3589905067 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa068073b90 con 0x7fa04806be30 2024-04-24T22:44:09.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.076+0000 7fa065600640 1 -- 172.21.15.47:0/3589905067 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252944 (secure 0 0 0) 0x7fa068073b90 con 0x7fa04806be30 2024-04-24T22:44:09.084 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 -- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa04806be30 msgr2=0x7fa04806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa04806be30 0x7fa04806e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009a40 tx=0x7fa05c00b040 comp rx=0 tx=0).stop 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 -- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa06814d580 msgr2=0x7fa06814da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa06814d580 0x7fa06814da10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa060031000 tx=0x7fa0600046c0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 -- 172.21.15.47:0/3589905067 shutdown_connections 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa04806be30 0x7fa04806e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa06814d580 0x7fa06814da10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa06815e830 0x7fa068153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 --2- 172.21.15.47:0/3589905067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068154960 0x7fa068153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 -- 172.21.15.47:0/3589905067 >> 172.21.15.47:0/3589905067 conn(0x7fa06806e580 msgr2=0x7fa068158780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 -- 172.21.15.47:0/3589905067 shutdown_connections 2024-04-24T22:44:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.082+0000 7fa070000640 1 -- 172.21.15.47:0/3589905067 wait complete. 2024-04-24T22:44:09.106 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-04-24T22:44:08.703762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":981383,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":9788,"num_write_kb":25168823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11246,"ondisk_log_size":11246,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282584,"kb_used_data":4056,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754692136,"statfs":{"total":773094113280,"available":772804747264,"internally_reserved":0,"allocated":4153344,"data_stored":2665126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":285134387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":700,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003296"},"pg_stats":[{"pgid":"3.3f","version":"23'159","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.666893+0000","last_change":"2024-04-24T22:40:23.914828+0000","last_active":"2024-04-24T22:43:48.666893+0000","last_peered":"2024-04-24T22:43:48.666893+0000","last_clean":"2024-04-24T22:43:48.666893+0000","last_became_active":"2024-04-24T22:40:23.913848+0000","last_became_peered":"2024-04-24T22:40:23.913848+0000","last_unstale":"2024-04-24T22:43:48.666893+0000","last_undegraded":"2024-04-24T22:43:48.666893+0000","last_fullsized":"2024-04-24T22:43:48.666893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.690809+0000","last_change":"2024-04-24T22:39:37.697245+0000","last_active":"2024-04-24T22:43:48.690809+0000","last_peered":"2024-04-24T22:43:48.690809+0000","last_clean":"2024-04-24T22:43:48.690809+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:43:48.690809+0000","last_undegraded":"2024-04-24T22:43:48.690809+0000","last_fullsized":"2024-04-24T22:43:48.690809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.689091+0000","last_change":"2024-04-24T22:39:37.709539+0000","last_active":"2024-04-24T22:43:48.689091+0000","last_peered":"2024-04-24T22:43:48.689091+0000","last_clean":"2024-04-24T22:43:48.689091+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:43:48.689091+0000","last_undegraded":"2024-04-24T22:43:48.689091+0000","last_fullsized":"2024-04-24T22:43:48.689091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'185","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.691701+0000","last_change":"2024-04-24T22:39:37.719136+0000","last_active":"2024-04-24T22:43:48.691701+0000","last_peered":"2024-04-24T22:43:48.691701+0000","last_clean":"2024-04-24T22:43:48.691701+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:43:48.691701+0000","last_undegraded":"2024-04-24T22:43:48.691701+0000","last_fullsized":"2024-04-24T22:43:48.691701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.667187+0000","last_change":"2024-04-24T22:39:37.704671+0000","last_active":"2024-04-24T22:43:48.667187+0000","last_peered":"2024-04-24T22:43:48.667187+0000","last_clean":"2024-04-24T22:43:48.667187+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:43:48.667187+0000","last_undegraded":"2024-04-24T22:43:48.667187+0000","last_fullsized":"2024-04-24T22:43:48.667187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.683504+0000","last_change":"2024-04-24T22:39:37.703880+0000","last_active":"2024-04-24T22:43:48.683504+0000","last_peered":"2024-04-24T22:43:48.683504+0000","last_clean":"2024-04-24T22:43:48.683504+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:43:48.683504+0000","last_undegraded":"2024-04-24T22:43:48.683504+0000","last_fullsized":"2024-04-24T22:43:48.683504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.627153+0000","last_change":"2024-04-24T22:39:37.703539+0000","last_active":"2024-04-24T22:43:48.627153+0000","last_peered":"2024-04-24T22:43:48.627153+0000","last_clean":"2024-04-24T22:43:48.627153+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:43:48.627153+0000","last_undegraded":"2024-04-24T22:43:48.627153+0000","last_fullsized":"2024-04-24T22:43:48.627153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'195","reported_seq":212,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.682546+0000","last_change":"2024-04-24T22:39:37.713543+0000","last_active":"2024-04-24T22:43:48.682546+0000","last_peered":"2024-04-24T22:43:48.682546+0000","last_clean":"2024-04-24T22:43:48.682546+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:43:48.682546+0000","last_undegraded":"2024-04-24T22:43:48.682546+0000","last_fullsized":"2024-04-24T22:43:48.682546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'167","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.645645+0000","last_change":"2024-04-24T22:39:37.706715+0000","last_active":"2024-04-24T22:43:48.645645+0000","last_peered":"2024-04-24T22:43:48.645645+0000","last_clean":"2024-04-24T22:43:48.645645+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:43:48.645645+0000","last_undegraded":"2024-04-24T22:43:48.645645+0000","last_fullsized":"2024-04-24T22:43:48.645645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'175","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.672260+0000","last_change":"2024-04-24T22:40:23.915517+0000","last_active":"2024-04-24T22:43:48.672260+0000","last_peered":"2024-04-24T22:43:48.672260+0000","last_clean":"2024-04-24T22:43:48.672260+0000","last_became_active":"2024-04-24T22:40:23.915141+0000","last_became_peered":"2024-04-24T22:40:23.915141+0000","last_unstale":"2024-04-24T22:43:48.672260+0000","last_undegraded":"2024-04-24T22:43:48.672260+0000","last_fullsized":"2024-04-24T22:43:48.672260+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:07.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'208","reported_seq":225,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.677013+0000","last_change":"2024-04-24T22:39:37.701069+0000","last_active":"2024-04-24T22:43:48.677013+0000","last_peered":"2024-04-24T22:43:48.677013+0000","last_clean":"2024-04-24T22:43:48.677013+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:43:48.677013+0000","last_undegraded":"2024-04-24T22:43:48.677013+0000","last_fullsized":"2024-04-24T22:43:48.677013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.686235+0000","last_change":"2024-04-24T22:39:37.693508+0000","last_active":"2024-04-24T22:43:48.686235+0000","last_peered":"2024-04-24T22:43:48.686235+0000","last_clean":"2024-04-24T22:43:48.686235+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:43:48.686235+0000","last_undegraded":"2024-04-24T22:43:48.686235+0000","last_fullsized":"2024-04-24T22:43:48.686235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'167","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.669776+0000","last_change":"2024-04-24T22:39:37.705157+0000","last_active":"2024-04-24T22:43:48.669776+0000","last_peered":"2024-04-24T22:43:48.669776+0000","last_clean":"2024-04-24T22:43:48.669776+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:43:48.669776+0000","last_undegraded":"2024-04-24T22:43:48.669776+0000","last_fullsized":"2024-04-24T22:43:48.669776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'164","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.622338+0000","last_change":"2024-04-24T22:39:37.698333+0000","last_active":"2024-04-24T22:43:48.622338+0000","last_peered":"2024-04-24T22:43:48.622338+0000","last_clean":"2024-04-24T22:43:48.622338+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:43:48.622338+0000","last_undegraded":"2024-04-24T22:43:48.622338+0000","last_fullsized":"2024-04-24T22:43:48.622338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:57.510197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.671167+0000","last_change":"2024-04-24T22:39:37.696874+0000","last_active":"2024-04-24T22:43:48.671167+0000","last_peered":"2024-04-24T22:43:48.671167+0000","last_clean":"2024-04-24T22:43:48.671167+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:43:48.671167+0000","last_undegraded":"2024-04-24T22:43:48.671167+0000","last_fullsized":"2024-04-24T22:43:48.671167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.654424+0000","last_change":"2024-04-24T22:39:37.703446+0000","last_active":"2024-04-24T22:43:48.654424+0000","last_peered":"2024-04-24T22:43:48.654424+0000","last_clean":"2024-04-24T22:43:48.654424+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:43:48.654424+0000","last_undegraded":"2024-04-24T22:43:48.654424+0000","last_fullsized":"2024-04-24T22:43:48.654424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'168","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.666782+0000","last_change":"2024-04-24T22:40:23.916129+0000","last_active":"2024-04-24T22:43:48.666782+0000","last_peered":"2024-04-24T22:43:48.666782+0000","last_clean":"2024-04-24T22:43:48.666782+0000","last_became_active":"2024-04-24T22:40:23.915441+0000","last_became_peered":"2024-04-24T22:40:23.915441+0000","last_unstale":"2024-04-24T22:43:48.666782+0000","last_undegraded":"2024-04-24T22:43:48.666782+0000","last_fullsized":"2024-04-24T22:43:48.666782+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'169","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.672272+0000","last_change":"2024-04-24T22:39:37.703899+0000","last_active":"2024-04-24T22:43:48.672272+0000","last_peered":"2024-04-24T22:43:48.672272+0000","last_clean":"2024-04-24T22:43:48.672272+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:43:48.672272+0000","last_undegraded":"2024-04-24T22:43:48.672272+0000","last_fullsized":"2024-04-24T22:43:48.672272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:32.758971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'198","reported_seq":215,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.680661+0000","last_change":"2024-04-24T22:39:37.711019+0000","last_active":"2024-04-24T22:43:48.680661+0000","last_peered":"2024-04-24T22:43:48.680661+0000","last_clean":"2024-04-24T22:43:48.680661+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:43:48.680661+0000","last_undegraded":"2024-04-24T22:43:48.680661+0000","last_fullsized":"2024-04-24T22:43:48.680661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'171","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.685173+0000","last_change":"2024-04-24T22:39:37.696214+0000","last_active":"2024-04-24T22:43:48.685173+0000","last_peered":"2024-04-24T22:43:48.685173+0000","last_clean":"2024-04-24T22:43:48.685173+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:43:48.685173+0000","last_undegraded":"2024-04-24T22:43:48.685173+0000","last_fullsized":"2024-04-24T22:43:48.685173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'143","reported_seq":160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.654913+0000","last_change":"2024-04-24T22:39:37.700151+0000","last_active":"2024-04-24T22:43:48.654913+0000","last_peered":"2024-04-24T22:43:48.654913+0000","last_clean":"2024-04-24T22:43:48.654913+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:43:48.654913+0000","last_undegraded":"2024-04-24T22:43:48.654913+0000","last_fullsized":"2024-04-24T22:43:48.654913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'189","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.664027+0000","last_change":"2024-04-24T22:39:37.705083+0000","last_active":"2024-04-24T22:43:48.664027+0000","last_peered":"2024-04-24T22:43:48.664027+0000","last_clean":"2024-04-24T22:43:48.664027+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:43:48.664027+0000","last_undegraded":"2024-04-24T22:43:48.664027+0000","last_fullsized":"2024-04-24T22:43:48.664027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'142","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.652343+0000","last_change":"2024-04-24T22:39:37.711193+0000","last_active":"2024-04-24T22:43:48.652343+0000","last_peered":"2024-04-24T22:43:48.652343+0000","last_clean":"2024-04-24T22:43:48.652343+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:43:48.652343+0000","last_undegraded":"2024-04-24T22:43:48.652343+0000","last_fullsized":"2024-04-24T22:43:48.652343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'201","reported_seq":219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.685527+0000","last_change":"2024-04-24T22:39:37.713910+0000","last_active":"2024-04-24T22:43:48.685527+0000","last_peered":"2024-04-24T22:43:48.685527+0000","last_clean":"2024-04-24T22:43:48.685527+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:43:48.685527+0000","last_undegraded":"2024-04-24T22:43:48.685527+0000","last_fullsized":"2024-04-24T22:43:48.685527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'158","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.666804+0000","last_change":"2024-04-24T22:39:37.717191+0000","last_active":"2024-04-24T22:43:48.666804+0000","last_peered":"2024-04-24T22:43:48.666804+0000","last_clean":"2024-04-24T22:43:48.666804+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:43:48.666804+0000","last_undegraded":"2024-04-24T22:43:48.666804+0000","last_fullsized":"2024-04-24T22:43:48.666804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'202","reported_seq":220,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.638120+0000","last_change":"2024-04-24T22:39:37.702983+0000","last_active":"2024-04-24T22:43:48.638120+0000","last_peered":"2024-04-24T22:43:48.638120+0000","last_clean":"2024-04-24T22:43:48.638120+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:43:48.638120+0000","last_undegraded":"2024-04-24T22:43:48.638120+0000","last_fullsized":"2024-04-24T22:43:48.638120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.682048+0000","last_change":"2024-04-24T22:39:37.714111+0000","last_active":"2024-04-24T22:43:48.682048+0000","last_peered":"2024-04-24T22:43:48.682048+0000","last_clean":"2024-04-24T22:43:48.682048+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:43:48.682048+0000","last_undegraded":"2024-04-24T22:43:48.682048+0000","last_fullsized":"2024-04-24T22:43:48.682048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.669969+0000","last_change":"2024-04-24T22:39:37.714877+0000","last_active":"2024-04-24T22:43:48.669969+0000","last_peered":"2024-04-24T22:43:48.669969+0000","last_clean":"2024-04-24T22:43:48.669969+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:43:48.669969+0000","last_undegraded":"2024-04-24T22:43:48.669969+0000","last_fullsized":"2024-04-24T22:43:48.669969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'127","reported_seq":145,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.674841+0000","last_change":"2024-04-24T22:39:37.714695+0000","last_active":"2024-04-24T22:43:48.674841+0000","last_peered":"2024-04-24T22:43:48.674841+0000","last_clean":"2024-04-24T22:43:48.674841+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:43:48.674841+0000","last_undegraded":"2024-04-24T22:43:48.674841+0000","last_fullsized":"2024-04-24T22:43:48.674841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'145","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.656139+0000","last_change":"2024-04-24T22:39:37.719187+0000","last_active":"2024-04-24T22:43:48.656139+0000","last_peered":"2024-04-24T22:43:48.656139+0000","last_clean":"2024-04-24T22:43:48.656139+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:43:48.656139+0000","last_undegraded":"2024-04-24T22:43:48.656139+0000","last_fullsized":"2024-04-24T22:43:48.656139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'146","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.688744+0000","last_change":"2024-04-24T22:39:37.707283+0000","last_active":"2024-04-24T22:43:48.688744+0000","last_peered":"2024-04-24T22:43:48.688744+0000","last_clean":"2024-04-24T22:43:48.688744+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:43:48.688744+0000","last_undegraded":"2024-04-24T22:43:48.688744+0000","last_fullsized":"2024-04-24T22:43:48.688744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.683443+0000","last_change":"2024-04-24T22:39:37.697902+0000","last_active":"2024-04-24T22:43:48.683443+0000","last_peered":"2024-04-24T22:43:48.683443+0000","last_clean":"2024-04-24T22:43:48.683443+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:43:48.683443+0000","last_undegraded":"2024-04-24T22:43:48.683443+0000","last_fullsized":"2024-04-24T22:43:48.683443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'142","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.654469+0000","last_change":"2024-04-24T22:39:37.705667+0000","last_active":"2024-04-24T22:43:48.654469+0000","last_peered":"2024-04-24T22:43:48.654469+0000","last_clean":"2024-04-24T22:43:48.654469+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:43:48.654469+0000","last_undegraded":"2024-04-24T22:43:48.654469+0000","last_fullsized":"2024-04-24T22:43:48.654469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:50.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:24.451020+0000","last_change":"2024-04-24T22:39:36.707990+0000","last_active":"2024-04-24T22:40:24.451020+0000","last_peered":"2024-04-24T22:40:24.451020+0000","last_clean":"2024-04-24T22:40:24.451020+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:40:24.451020+0000","last_undegraded":"2024-04-24T22:40:24.451020+0000","last_fullsized":"2024-04-24T22:40:24.451020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'154","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.622757+0000","last_change":"2024-04-24T22:39:37.700406+0000","last_active":"2024-04-24T22:43:48.622757+0000","last_peered":"2024-04-24T22:43:48.622757+0000","last_clean":"2024-04-24T22:43:48.622757+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:43:48.622757+0000","last_undegraded":"2024-04-24T22:43:48.622757+0000","last_fullsized":"2024-04-24T22:43:48.622757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.908485+0000","last_change":"2024-04-24T22:39:36.701398+0000","last_active":"2024-04-24T22:40:23.908485+0000","last_peered":"2024-04-24T22:40:23.908485+0000","last_clean":"2024-04-24T22:40:23.908485+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:40:23.908485+0000","last_undegraded":"2024-04-24T22:40:23.908485+0000","last_fullsized":"2024-04-24T22:40:23.908485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:24.450888+0000","last_change":"2024-04-24T22:39:36.712294+0000","last_active":"2024-04-24T22:40:24.450888+0000","last_peered":"2024-04-24T22:40:24.450888+0000","last_clean":"2024-04-24T22:40:24.450888+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:40:24.450888+0000","last_undegraded":"2024-04-24T22:40:24.450888+0000","last_fullsized":"2024-04-24T22:40:24.450888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'182","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673441+0000","last_change":"2024-04-24T22:40:23.910148+0000","last_active":"2024-04-24T22:43:48.673441+0000","last_peered":"2024-04-24T22:43:48.673441+0000","last_clean":"2024-04-24T22:43:48.673441+0000","last_became_active":"2024-04-24T22:40:23.909573+0000","last_became_peered":"2024-04-24T22:40:23.909573+0000","last_unstale":"2024-04-24T22:43:48.673441+0000","last_undegraded":"2024-04-24T22:43:48.673441+0000","last_fullsized":"2024-04-24T22:43:48.673441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:07.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906152+0000","last_change":"2024-04-24T22:39:36.698983+0000","last_active":"2024-04-24T22:40:23.906152+0000","last_peered":"2024-04-24T22:40:23.906152+0000","last_clean":"2024-04-24T22:40:23.906152+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:40:23.906152+0000","last_undegraded":"2024-04-24T22:40:23.906152+0000","last_fullsized":"2024-04-24T22:40:23.906152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'177","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.676625+0000","last_change":"2024-04-24T22:39:37.714229+0000","last_active":"2024-04-24T22:43:48.676625+0000","last_peered":"2024-04-24T22:43:48.676625+0000","last_clean":"2024-04-24T22:43:48.676625+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:43:48.676625+0000","last_undegraded":"2024-04-24T22:43:48.676625+0000","last_fullsized":"2024-04-24T22:43:48.676625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:04.691585+0000","last_change":"2024-04-24T22:39:36.705776+0000","last_active":"2024-04-24T22:42:04.691585+0000","last_peered":"2024-04-24T22:42:04.691585+0000","last_clean":"2024-04-24T22:42:04.691585+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:42:04.691585+0000","last_undegraded":"2024-04-24T22:42:04.691585+0000","last_fullsized":"2024-04-24T22:42:04.691585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'241","reported_seq":258,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673855+0000","last_change":"2024-04-24T22:39:37.701436+0000","last_active":"2024-04-24T22:43:48.673855+0000","last_peered":"2024-04-24T22:43:48.673855+0000","last_clean":"2024-04-24T22:43:48.673855+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:43:48.673855+0000","last_undegraded":"2024-04-24T22:43:48.673855+0000","last_fullsized":"2024-04-24T22:43:48.673855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906331+0000","last_change":"2024-04-24T22:39:36.700237+0000","last_active":"2024-04-24T22:40:23.906331+0000","last_peered":"2024-04-24T22:40:23.906331+0000","last_clean":"2024-04-24T22:40:23.906331+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:40:23.906331+0000","last_undegraded":"2024-04-24T22:40:23.906331+0000","last_fullsized":"2024-04-24T22:40:23.906331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'151","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.660220+0000","last_change":"2024-04-24T22:39:37.703532+0000","last_active":"2024-04-24T22:43:48.660220+0000","last_peered":"2024-04-24T22:43:48.660220+0000","last_clean":"2024-04-24T22:43:48.660220+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:43:48.660220+0000","last_undegraded":"2024-04-24T22:43:48.660220+0000","last_fullsized":"2024-04-24T22:43:48.660220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'11","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:53.085629+0000","last_change":"2024-04-24T22:39:36.717477+0000","last_active":"2024-04-24T22:43:53.085629+0000","last_peered":"2024-04-24T22:43:53.085629+0000","last_clean":"2024-04-24T22:43:53.085629+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:43:53.085629+0000","last_undegraded":"2024-04-24T22:43:53.085629+0000","last_fullsized":"2024-04-24T22:43:53.085629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'157","reported_seq":174,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.640565+0000","last_change":"2024-04-24T22:39:37.699255+0000","last_active":"2024-04-24T22:43:48.640565+0000","last_peered":"2024-04-24T22:43:48.640565+0000","last_clean":"2024-04-24T22:43:48.640565+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:43:48.640565+0000","last_undegraded":"2024-04-24T22:43:48.640565+0000","last_fullsized":"2024-04-24T22:43:48.640565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:34.377878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'166","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:53.750307+0000","last_change":"2024-04-24T22:39:36.706567+0000","last_active":"2024-04-24T22:43:53.750307+0000","last_peered":"2024-04-24T22:43:53.750307+0000","last_clean":"2024-04-24T22:43:53.750307+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:43:53.750307+0000","last_undegraded":"2024-04-24T22:43:53.750307+0000","last_fullsized":"2024-04-24T22:43:53.750307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383830,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.670888+0000","last_change":"2024-04-24T22:39:37.712064+0000","last_active":"2024-04-24T22:43:48.670888+0000","last_peered":"2024-04-24T22:43:48.670888+0000","last_clean":"2024-04-24T22:43:48.670888+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:43:48.670888+0000","last_undegraded":"2024-04-24T22:43:48.670888+0000","last_fullsized":"2024-04-24T22:43:48.670888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'9","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.903984+0000","last_change":"2024-04-24T22:39:36.700041+0000","last_active":"2024-04-24T22:40:23.903984+0000","last_peered":"2024-04-24T22:40:23.903984+0000","last_clean":"2024-04-24T22:40:23.903984+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:40:23.903984+0000","last_undegraded":"2024-04-24T22:40:23.903984+0000","last_fullsized":"2024-04-24T22:40:23.903984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:50.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'168","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.624474+0000","last_change":"2024-04-24T22:39:37.691459+0000","last_active":"2024-04-24T22:43:48.624474+0000","last_peered":"2024-04-24T22:43:48.624474+0000","last_clean":"2024-04-24T22:43:48.624474+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:43:48.624474+0000","last_undegraded":"2024-04-24T22:43:48.624474+0000","last_fullsized":"2024-04-24T22:43:48.624474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906390+0000","last_change":"2024-04-24T22:39:36.706454+0000","last_active":"2024-04-24T22:40:23.906390+0000","last_peered":"2024-04-24T22:40:23.906390+0000","last_clean":"2024-04-24T22:40:23.906390+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:40:23.906390+0000","last_undegraded":"2024-04-24T22:40:23.906390+0000","last_fullsized":"2024-04-24T22:40:23.906390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'186","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.675788+0000","last_change":"2024-04-24T22:39:37.696724+0000","last_active":"2024-04-24T22:43:48.675788+0000","last_peered":"2024-04-24T22:43:48.675788+0000","last_clean":"2024-04-24T22:43:48.675788+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:43:48.675788+0000","last_undegraded":"2024-04-24T22:43:48.675788+0000","last_fullsized":"2024-04-24T22:43:48.675788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.908547+0000","last_change":"2024-04-24T22:39:36.701340+0000","last_active":"2024-04-24T22:40:23.908547+0000","last_peered":"2024-04-24T22:40:23.908547+0000","last_clean":"2024-04-24T22:40:23.908547+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:40:23.908547+0000","last_undegraded":"2024-04-24T22:40:23.908547+0000","last_fullsized":"2024-04-24T22:40:23.908547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'191","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.679977+0000","last_change":"2024-04-24T22:39:37.697532+0000","last_active":"2024-04-24T22:43:48.679977+0000","last_peered":"2024-04-24T22:43:48.679977+0000","last_clean":"2024-04-24T22:43:48.679977+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:43:48.679977+0000","last_undegraded":"2024-04-24T22:43:48.679977+0000","last_fullsized":"2024-04-24T22:43:48.679977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'18","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:59.706381+0000","last_change":"2024-04-24T22:39:36.724649+0000","last_active":"2024-04-24T22:41:59.706381+0000","last_peered":"2024-04-24T22:41:59.706381+0000","last_clean":"2024-04-24T22:41:59.706381+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:41:59.706381+0000","last_undegraded":"2024-04-24T22:41:59.706381+0000","last_fullsized":"2024-04-24T22:41:59.706381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'209","reported_seq":226,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.637954+0000","last_change":"2024-04-24T22:39:37.702890+0000","last_active":"2024-04-24T22:43:48.637954+0000","last_peered":"2024-04-24T22:43:48.637954+0000","last_clean":"2024-04-24T22:43:48.637954+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:43:48.637954+0000","last_undegraded":"2024-04-24T22:43:48.637954+0000","last_fullsized":"2024-04-24T22:43:48.637954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:24.792909+0000","last_change":"2024-04-24T22:39:36.716044+0000","last_active":"2024-04-24T22:40:24.792909+0000","last_peered":"2024-04-24T22:40:24.792909+0000","last_clean":"2024-04-24T22:40:24.792909+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:40:24.792909+0000","last_undegraded":"2024-04-24T22:40:24.792909+0000","last_fullsized":"2024-04-24T22:40:24.792909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:18:40.023933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'170","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.692442+0000","last_change":"2024-04-24T22:39:37.702621+0000","last_active":"2024-04-24T22:43:48.692442+0000","last_peered":"2024-04-24T22:43:48.692442+0000","last_clean":"2024-04-24T22:43:48.692442+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:43:48.692442+0000","last_undegraded":"2024-04-24T22:43:48.692442+0000","last_fullsized":"2024-04-24T22:43:48.692442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.915496+0000","last_change":"2024-04-24T22:39:36.714272+0000","last_active":"2024-04-24T22:40:23.915496+0000","last_peered":"2024-04-24T22:40:23.915496+0000","last_clean":"2024-04-24T22:40:23.915496+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:40:23.915496+0000","last_undegraded":"2024-04-24T22:40:23.915496+0000","last_fullsized":"2024-04-24T22:40:23.915496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.619238+0000","last_change":"2024-04-24T22:39:37.703786+0000","last_active":"2024-04-24T22:43:48.619238+0000","last_peered":"2024-04-24T22:43:48.619238+0000","last_clean":"2024-04-24T22:43:48.619238+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:43:48.619238+0000","last_undegraded":"2024-04-24T22:43:48.619238+0000","last_fullsized":"2024-04-24T22:43:48.619238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:55.990656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.907195+0000","last_change":"2024-04-24T22:39:36.715437+0000","last_active":"2024-04-24T22:40:23.907195+0000","last_peered":"2024-04-24T22:40:23.907195+0000","last_clean":"2024-04-24T22:40:23.907195+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:40:23.907195+0000","last_undegraded":"2024-04-24T22:40:23.907195+0000","last_fullsized":"2024-04-24T22:40:23.907195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.681549+0000","last_change":"2024-04-24T22:39:37.715801+0000","last_active":"2024-04-24T22:43:48.681549+0000","last_peered":"2024-04-24T22:43:48.681549+0000","last_clean":"2024-04-24T22:43:48.681549+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:43:48.681549+0000","last_undegraded":"2024-04-24T22:43:48.681549+0000","last_fullsized":"2024-04-24T22:43:48.681549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.915114+0000","last_change":"2024-04-24T22:39:36.710422+0000","last_active":"2024-04-24T22:40:23.915114+0000","last_peered":"2024-04-24T22:40:23.915114+0000","last_clean":"2024-04-24T22:40:23.915114+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:40:23.915114+0000","last_undegraded":"2024-04-24T22:40:23.915114+0000","last_fullsized":"2024-04-24T22:40:23.915114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'168","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.663298+0000","last_change":"2024-04-24T22:39:37.706745+0000","last_active":"2024-04-24T22:43:48.663298+0000","last_peered":"2024-04-24T22:43:48.663298+0000","last_clean":"2024-04-24T22:43:48.663298+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:43:48.663298+0000","last_undegraded":"2024-04-24T22:43:48.663298+0000","last_fullsized":"2024-04-24T22:43:48.663298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:34.691489+0000","last_change":"2024-04-24T22:39:32.657965+0000","last_active":"2024-04-24T22:41:34.691489+0000","last_peered":"2024-04-24T22:41:34.691489+0000","last_clean":"2024-04-24T22:41:34.691489+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:41:34.691489+0000","last_undegraded":"2024-04-24T22:41:34.691489+0000","last_fullsized":"2024-04-24T22:41:34.691489+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_clean_scrub_stamp":"2024-04-24T22:39:31.635804+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.714153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.694937+0000","last_change":"2024-04-24T22:39:36.710381+0000","last_active":"2024-04-24T22:41:44.694937+0000","last_peered":"2024-04-24T22:41:44.694937+0000","last_clean":"2024-04-24T22:41:44.694937+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:41:44.694937+0000","last_undegraded":"2024-04-24T22:41:44.694937+0000","last_fullsized":"2024-04-24T22:41:44.694937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.662348+0000","last_change":"2024-04-24T22:39:37.713956+0000","last_active":"2024-04-24T22:43:48.662348+0000","last_peered":"2024-04-24T22:43:48.662348+0000","last_clean":"2024-04-24T22:43:48.662348+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:43:48.662348+0000","last_undegraded":"2024-04-24T22:43:48.662348+0000","last_fullsized":"2024-04-24T22:43:48.662348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:39.691975+0000","last_change":"2024-04-24T22:39:36.715290+0000","last_active":"2024-04-24T22:41:39.691975+0000","last_peered":"2024-04-24T22:41:39.691975+0000","last_clean":"2024-04-24T22:41:39.691975+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:41:39.691975+0000","last_undegraded":"2024-04-24T22:41:39.691975+0000","last_fullsized":"2024-04-24T22:41:39.691975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.692616+0000","last_change":"2024-04-24T22:39:37.695419+0000","last_active":"2024-04-24T22:43:48.692616+0000","last_peered":"2024-04-24T22:43:48.692616+0000","last_clean":"2024-04-24T22:43:48.692616+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:43:48.692616+0000","last_undegraded":"2024-04-24T22:43:48.692616+0000","last_fullsized":"2024-04-24T22:43:48.692616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:08.784309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:58.090196+0000","last_change":"2024-04-24T22:39:36.707548+0000","last_active":"2024-04-24T22:43:58.090196+0000","last_peered":"2024-04-24T22:43:58.090196+0000","last_clean":"2024-04-24T22:43:58.090196+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:43:58.090196+0000","last_undegraded":"2024-04-24T22:43:58.090196+0000","last_fullsized":"2024-04-24T22:43:58.090196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673312+0000","last_change":"2024-04-24T22:39:37.703301+0000","last_active":"2024-04-24T22:43:48.673312+0000","last_peered":"2024-04-24T22:43:48.673312+0000","last_clean":"2024-04-24T22:43:48.673312+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:43:48.673312+0000","last_undegraded":"2024-04-24T22:43:48.673312+0000","last_fullsized":"2024-04-24T22:43:48.673312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.694926+0000","last_change":"2024-04-24T22:39:36.713486+0000","last_active":"2024-04-24T22:41:44.694926+0000","last_peered":"2024-04-24T22:41:44.694926+0000","last_clean":"2024-04-24T22:41:44.694926+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:41:44.694926+0000","last_undegraded":"2024-04-24T22:41:44.694926+0000","last_fullsized":"2024-04-24T22:41:44.694926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.694657+0000","last_change":"2024-04-24T22:39:37.705705+0000","last_active":"2024-04-24T22:43:48.694657+0000","last_peered":"2024-04-24T22:43:48.694657+0000","last_clean":"2024-04-24T22:43:48.694657+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:43:48.694657+0000","last_undegraded":"2024-04-24T22:43:48.694657+0000","last_fullsized":"2024-04-24T22:43:48.694657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:34.424877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.703126+0000","last_change":"2024-04-24T22:39:36.723125+0000","last_active":"2024-04-24T22:41:44.703126+0000","last_peered":"2024-04-24T22:41:44.703126+0000","last_clean":"2024-04-24T22:41:44.703126+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:41:44.703126+0000","last_undegraded":"2024-04-24T22:41:44.703126+0000","last_fullsized":"2024-04-24T22:41:44.703126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'155","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.676764+0000","last_change":"2024-04-24T22:39:37.713947+0000","last_active":"2024-04-24T22:43:48.676764+0000","last_peered":"2024-04-24T22:43:48.676764+0000","last_clean":"2024-04-24T22:43:48.676764+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:43:48.676764+0000","last_undegraded":"2024-04-24T22:43:48.676764+0000","last_fullsized":"2024-04-24T22:43:48.676764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.692421+0000","last_change":"2024-04-24T22:39:36.716541+0000","last_active":"2024-04-24T22:41:44.692421+0000","last_peered":"2024-04-24T22:41:44.692421+0000","last_clean":"2024-04-24T22:41:44.692421+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:41:44.692421+0000","last_undegraded":"2024-04-24T22:41:44.692421+0000","last_fullsized":"2024-04-24T22:41:44.692421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:33.366773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.694417+0000","last_change":"2024-04-24T22:39:37.708069+0000","last_active":"2024-04-24T22:43:48.694417+0000","last_peered":"2024-04-24T22:43:48.694417+0000","last_clean":"2024-04-24T22:43:48.694417+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:43:48.694417+0000","last_undegraded":"2024-04-24T22:43:48.694417+0000","last_fullsized":"2024-04-24T22:43:48.694417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'27","reported_seq":47,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:57.792124+0000","last_change":"2024-04-24T22:39:36.718740+0000","last_active":"2024-04-24T22:43:57.792124+0000","last_peered":"2024-04-24T22:43:57.792124+0000","last_clean":"2024-04-24T22:43:57.792124+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:43:57.792124+0000","last_undegraded":"2024-04-24T22:43:57.792124+0000","last_fullsized":"2024-04-24T22:43:57.792124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'161","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.679380+0000","last_change":"2024-04-24T22:39:37.704496+0000","last_active":"2024-04-24T22:43:48.679380+0000","last_peered":"2024-04-24T22:43:48.679380+0000","last_clean":"2024-04-24T22:43:48.679380+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:43:48.679380+0000","last_undegraded":"2024-04-24T22:43:48.679380+0000","last_fullsized":"2024-04-24T22:43:48.679380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:53.082764+0000","last_change":"2024-04-24T22:39:36.701308+0000","last_active":"2024-04-24T22:43:53.082764+0000","last_peered":"2024-04-24T22:43:53.082764+0000","last_clean":"2024-04-24T22:43:53.082764+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:43:53.082764+0000","last_undegraded":"2024-04-24T22:43:53.082764+0000","last_fullsized":"2024-04-24T22:43:53.082764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.683485+0000","last_change":"2024-04-24T22:39:37.700189+0000","last_active":"2024-04-24T22:43:48.683485+0000","last_peered":"2024-04-24T22:43:48.683485+0000","last_clean":"2024-04-24T22:43:48.683485+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:43:48.683485+0000","last_undegraded":"2024-04-24T22:43:48.683485+0000","last_fullsized":"2024-04-24T22:43:48.683485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:49.693021+0000","last_change":"2024-04-24T22:39:36.716460+0000","last_active":"2024-04-24T22:41:49.693021+0000","last_peered":"2024-04-24T22:41:49.693021+0000","last_clean":"2024-04-24T22:41:49.693021+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:41:49.693021+0000","last_undegraded":"2024-04-24T22:41:49.693021+0000","last_fullsized":"2024-04-24T22:41:49.693021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.693303+0000","last_change":"2024-04-24T22:39:37.704143+0000","last_active":"2024-04-24T22:43:48.693303+0000","last_peered":"2024-04-24T22:43:48.693303+0000","last_clean":"2024-04-24T22:43:48.693303+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:43:48.693303+0000","last_undegraded":"2024-04-24T22:43:48.693303+0000","last_fullsized":"2024-04-24T22:43:48.693303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:58.556519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:49.703968+0000","last_change":"2024-04-24T22:39:36.721540+0000","last_active":"2024-04-24T22:41:49.703968+0000","last_peered":"2024-04-24T22:41:49.703968+0000","last_clean":"2024-04-24T22:41:49.703968+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:41:49.703968+0000","last_undegraded":"2024-04-24T22:41:49.703968+0000","last_fullsized":"2024-04-24T22:41:49.703968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'164","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.664232+0000","last_change":"2024-04-24T22:39:37.714150+0000","last_active":"2024-04-24T22:43:48.664232+0000","last_peered":"2024-04-24T22:43:48.664232+0000","last_clean":"2024-04-24T22:43:48.664232+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:43:48.664232+0000","last_undegraded":"2024-04-24T22:43:48.664232+0000","last_fullsized":"2024-04-24T22:43:48.664232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:49.704015+0000","last_change":"2024-04-24T22:39:36.716667+0000","last_active":"2024-04-24T22:41:49.704015+0000","last_peered":"2024-04-24T22:41:49.704015+0000","last_clean":"2024-04-24T22:41:49.704015+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:41:49.704015+0000","last_undegraded":"2024-04-24T22:41:49.704015+0000","last_fullsized":"2024-04-24T22:41:49.704015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.675064+0000","last_change":"2024-04-24T22:39:37.715906+0000","last_active":"2024-04-24T22:43:48.675064+0000","last_peered":"2024-04-24T22:43:48.675064+0000","last_clean":"2024-04-24T22:43:48.675064+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:43:48.675064+0000","last_undegraded":"2024-04-24T22:43:48.675064+0000","last_fullsized":"2024-04-24T22:43:48.675064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:54.693418+0000","last_change":"2024-04-24T22:39:36.698212+0000","last_active":"2024-04-24T22:41:54.693418+0000","last_peered":"2024-04-24T22:41:54.693418+0000","last_clean":"2024-04-24T22:41:54.693418+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:41:54.693418+0000","last_undegraded":"2024-04-24T22:41:54.693418+0000","last_fullsized":"2024-04-24T22:41:54.693418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'149","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.686905+0000","last_change":"2024-04-24T22:39:37.696143+0000","last_active":"2024-04-24T22:43:48.686905+0000","last_peered":"2024-04-24T22:43:48.686905+0000","last_clean":"2024-04-24T22:43:48.686905+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:43:48.686905+0000","last_undegraded":"2024-04-24T22:43:48.686905+0000","last_fullsized":"2024-04-24T22:43:48.686905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'190","reported_seq":207,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.680917+0000","last_change":"2024-04-24T22:39:37.696113+0000","last_active":"2024-04-24T22:43:48.680917+0000","last_peered":"2024-04-24T22:43:48.680917+0000","last_clean":"2024-04-24T22:43:48.680917+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:43:48.680917+0000","last_undegraded":"2024-04-24T22:43:48.680917+0000","last_fullsized":"2024-04-24T22:43:48.680917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906632+0000","last_change":"2024-04-24T22:39:36.699149+0000","last_active":"2024-04-24T22:40:23.906632+0000","last_peered":"2024-04-24T22:40:23.906632+0000","last_clean":"2024-04-24T22:40:23.906632+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:40:23.906632+0000","last_undegraded":"2024-04-24T22:40:23.906632+0000","last_fullsized":"2024-04-24T22:40:23.906632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.665742+0000","last_change":"2024-04-24T22:39:37.700972+0000","last_active":"2024-04-24T22:43:48.665742+0000","last_peered":"2024-04-24T22:43:48.665742+0000","last_clean":"2024-04-24T22:43:48.665742+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:43:48.665742+0000","last_undegraded":"2024-04-24T22:43:48.665742+0000","last_fullsized":"2024-04-24T22:43:48.665742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'14","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:07.936964+0000","last_change":"2024-04-24T22:39:36.698567+0000","last_active":"2024-04-24T22:42:07.936964+0000","last_peered":"2024-04-24T22:42:07.936964+0000","last_clean":"2024-04-24T22:42:07.936964+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:42:07.936964+0000","last_undegraded":"2024-04-24T22:42:07.936964+0000","last_fullsized":"2024-04-24T22:42:07.936964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.661869+0000","last_change":"2024-04-24T22:39:37.706951+0000","last_active":"2024-04-24T22:43:48.661869+0000","last_peered":"2024-04-24T22:43:48.661869+0000","last_clean":"2024-04-24T22:43:48.661869+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:43:48.661869+0000","last_undegraded":"2024-04-24T22:43:48.661869+0000","last_fullsized":"2024-04-24T22:43:48.661869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:39.700252+0000","last_change":"2024-04-24T22:39:36.703823+0000","last_active":"2024-04-24T22:41:39.700252+0000","last_peered":"2024-04-24T22:41:39.700252+0000","last_clean":"2024-04-24T22:41:39.700252+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:41:39.700252+0000","last_undegraded":"2024-04-24T22:41:39.700252+0000","last_fullsized":"2024-04-24T22:41:39.700252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:48.673604+0000","last_change":"2024-04-24T22:39:37.712078+0000","last_active":"2024-04-24T22:43:48.673604+0000","last_peered":"2024-04-24T22:43:48.673604+0000","last_clean":"2024-04-24T22:43:48.673604+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:43:48.673604+0000","last_undegraded":"2024-04-24T22:43:48.673604+0000","last_fullsized":"2024-04-24T22:43:48.673604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'13","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.706607+0000","last_change":"2024-04-24T22:39:36.719643+0000","last_active":"2024-04-24T22:41:44.706607+0000","last_peered":"2024-04-24T22:41:44.706607+0000","last_clean":"2024-04-24T22:41:44.706607+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:41:44.706607+0000","last_undegraded":"2024-04-24T22:41:44.706607+0000","last_fullsized":"2024-04-24T22:41:44.706607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:34.607834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":391015,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":413,"num_write_kb":583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032192,"data_stored":782030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":374,"ondisk_log_size":374,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40348,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331492,"statfs":{"total":96636764160,"available":96595447808,"internally_reserved":0,"allocated":872448,"data_stored":680263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":651264,"data_stored":472415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":290816,"data_stored":91985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39756,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332084,"statfs":{"total":96636764160,"available":96596054016,"internally_reserved":0,"allocated":266240,"data_stored":89125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":659456,"data_stored":472954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":278528,"data_stored":89762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":270336,"data_stored":89585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":864256,"data_stored":679037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":385159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":384620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:44:09.110 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:44:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.364+0000 7fe7bbe00640 1 Processor -- start 2024-04-24T22:44:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.365+0000 7fe7bbe00640 1 -- start start 2024-04-24T22:44:09.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.366+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7bc15bae0 0x7fe7bc15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:09.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.366+0000 7fe7bbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7bc06bf80 0x7fe7bc06b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:09.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.366+0000 7fe7bbe00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7bc15e290 con 0x7fe7bc0c9e50 2024-04-24T22:44:09.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.366+0000 7fe7bbe00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe7bc15eb30 con 0x7fe7bc06bf80 2024-04-24T22:44:09.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.366+0000 7fe7bbe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe7bc0cc5d0 con 0x7fe7bc15bae0 2024-04-24T22:44:09.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.367+0000 7fe7bb400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7bc06bf80 0x7fe7bc06b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:09.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.367+0000 7fe7ba000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7bc15bae0 0x7fe7bc15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:09.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.367+0000 7fe7bb400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7bc06bf80 0x7fe7bc06b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35614/0 (socket says 172.21.15.47:35614) 2024-04-24T22:44:09.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.367+0000 7fe7bb400640 1 -- 172.21.15.47:0/3490054858 learned_addr learned my addr 172.21.15.47:0/3490054858 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.367+0000 7fe7bb400640 1 -- 172.21.15.47:0/3490054858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7bc15bae0 msgr2=0x7fe7bc15dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.367+0000 7fe7bb400640 1 --2- 172.21.15.47:0/3490054858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7bc15bae0 0x7fe7bc15dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.367+0000 7fe7bb400640 1 -- 172.21.15.47:0/3490054858 >> v1:172.21.15.47:6789/0 conn(0x7fe7bc0c9e50 legacy=0x7fe7bc15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.367+0000 7fe7bb400640 1 -- 172.21.15.47:0/3490054858 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc15e570 con 0x7fe7bc06bf80 2024-04-24T22:44:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.368+0000 7fe7bb400640 1 --2- 172.21.15.47:0/3490054858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 0x7fe7bc06b160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009e80 tx=0x7fe7b402a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7df8261c67ec22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.369+0000 7fe7b9600640 1 -- 172.21.15.47:0/3490054858 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4036070 con 0x7fe7bc06bf80 2024-04-24T22:44:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.369+0000 7fe7b9600640 1 -- 172.21.15.47:0/3490054858 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b4031600 con 0x7fe7bc06bf80 2024-04-24T22:44:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.369+0000 7fe7b9600640 1 -- 172.21.15.47:0/3490054858 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40319d0 con 0x7fe7bc06bf80 2024-04-24T22:44:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.369+0000 7fe7bbe00640 1 -- 172.21.15.47:0/3490054858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 msgr2=0x7fe7bc06b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.369+0000 7fe7bbe00640 1 --2- 172.21.15.47:0/3490054858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 0x7fe7bc06b160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009e80 tx=0x7fe7b402a7e0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.370+0000 7fe7bbe00640 1 -- 172.21.15.47:0/3490054858 shutdown_connections 2024-04-24T22:44:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.370+0000 7fe7bbe00640 1 --2- 172.21.15.47:0/3490054858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 0x7fe7bc06b160 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.370+0000 7fe7bbe00640 1 --2- 172.21.15.47:0/3490054858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7bc15bae0 0x7fe7bc15dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.370+0000 7fe7bbe00640 1 -- 172.21.15.47:0/3490054858 >> 172.21.15.47:0/3490054858 conn(0x7fe7bc0cfd00 msgr2=0x7fe7bc0d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.370+0000 7fe7bbe00640 1 -- 172.21.15.47:0/3490054858 shutdown_connections 2024-04-24T22:44:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.370+0000 7fe7bbe00640 1 -- 172.21.15.47:0/3490054858 wait complete. 2024-04-24T22:44:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.371+0000 7fe7bbe00640 1 Processor -- start 2024-04-24T22:44:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.372+0000 7fe7bbe00640 1 -- start start 2024-04-24T22:44:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.372+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 0x7fe7bc1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.372+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7bc0c9e50 0x7fe7bc153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.372+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7bc15bae0 0x7fe7bc157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.372+0000 7fe7bbe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc15ab60 con 0x7fe7bc06bf80 2024-04-24T22:44:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.372+0000 7fe7bbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc15a880 con 0x7fe7bc0c9e50 2024-04-24T22:44:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7bbe00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe7bc06c430 con 0x7fe7bc15bae0 2024-04-24T22:44:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 0x7fe7bc1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7baa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 0x7fe7bc1595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35624/0 (socket says 172.21.15.47:35624) 2024-04-24T22:44:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7baa00640 1 -- 172.21.15.47:0/886026767 learned_addr learned my addr 172.21.15.47:0/886026767 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7ba000640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7bc0c9e50 0x7fe7bc153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7bb400640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7bc15bae0 0x7fe7bc157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7baa00640 1 -- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7bc15bae0 msgr2=0x7fe7bc157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7baa00640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7bc15bae0 0x7fe7bc157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7baa00640 1 -- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7bc0c9e50 msgr2=0x7fe7bc153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7baa00640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7bc0c9e50 0x7fe7bc153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.373+0000 7fe7baa00640 1 -- 172.21.15.47:0/886026767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc159b30 con 0x7fe7bc06bf80 2024-04-24T22:44:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.374+0000 7fe7baa00640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 0x7fe7bc1595f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c400 tx=0x7fe7ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.374+0000 7fe7b2a00640 1 -- 172.21.15.47:0/886026767 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0052d0 con 0x7fe7bc06bf80 2024-04-24T22:44:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.374+0000 7fe7b2a00640 1 -- 172.21.15.47:0/886026767 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ac005490 con 0x7fe7bc06bf80 2024-04-24T22:44:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.374+0000 7fe7b2a00640 1 -- 172.21.15.47:0/886026767 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac007260 con 0x7fe7bc06bf80 2024-04-24T22:44:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.374+0000 7fe7bbe00640 1 -- 172.21.15.47:0/886026767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7bc1f4230 con 0x7fe7bc06bf80 2024-04-24T22:44:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.374+0000 7fe7bbe00640 1 -- 172.21.15.47:0/886026767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7bc1f44b0 con 0x7fe7bc06bf80 2024-04-24T22:44:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.375+0000 7fe7bbe00640 1 -- 172.21.15.47:0/886026767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7bc04f5e0 con 0x7fe7bc06bf80 2024-04-24T22:44:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.376+0000 7fe7b2a00640 1 -- 172.21.15.47:0/886026767 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7ac007400 con 0x7fe7bc06bf80 2024-04-24T22:44:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.377+0000 7fe7b2a00640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe79806be60 0x7fe79806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.377+0000 7fe7b2a00640 1 -- 172.21.15.47:0/886026767 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe7ac08a460 con 0x7fe7bc06bf80 2024-04-24T22:44:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.378+0000 7fe7ba000640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe79806be60 0x7fe79806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.378+0000 7fe7ba000640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe79806be60 0x7fe79806e320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc15bf00 tx=0x7fe7a4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.382+0000 7fe7b2a00640 1 -- 172.21.15.47:0/886026767 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe7bc1f4230 con 0x7fe7bc06bf80 2024-04-24T22:44:09.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.589+0000 7fe7bbe00640 1 -- 172.21.15.47:0/886026767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7bc0cb760 con 0x7fe7bc06bf80 2024-04-24T22:44:09.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.590+0000 7fe7b2a00640 1 -- 172.21.15.47:0/886026767 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16843 (secure 0 0 0) 0x7fe7ac0586a0 con 0x7fe7bc06bf80 2024-04-24T22:44:09.593 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:09.593 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:40:23.897411+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:44:09.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.593+0000 7fe7bbe00640 1 -- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe79806be60 msgr2=0x7fe79806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:09.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.593+0000 7fe7bbe00640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe79806be60 0x7fe79806e320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc15bf00 tx=0x7fe7a4009190 comp rx=0 tx=0).stop 2024-04-24T22:44:09.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.593+0000 7fe7bbe00640 1 -- 172.21.15.47:0/886026767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 msgr2=0x7fe7bc1595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:09.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.593+0000 7fe7bbe00640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 0x7fe7bc1595f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c400 tx=0x7fe7ac00c8c0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.593+0000 7fe7bbe00640 1 -- 172.21.15.47:0/886026767 shutdown_connections 2024-04-24T22:44:09.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.593+0000 7fe7bbe00640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe79806be60 0x7fe79806e320 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.593+0000 7fe7bbe00640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7bc15bae0 0x7fe7bc157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.593+0000 7fe7bbe00640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7bc0c9e50 0x7fe7bc153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.593+0000 7fe7bbe00640 1 --2- 172.21.15.47:0/886026767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06bf80 0x7fe7bc1595f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:09.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.593+0000 7fe7bbe00640 1 -- 172.21.15.47:0/886026767 >> 172.21.15.47:0/886026767 conn(0x7fe7bc0cfd00 msgr2=0x7fe7bc06b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:09.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.594+0000 7fe7bbe00640 1 -- 172.21.15.47:0/886026767 shutdown_connections 2024-04-24T22:44:09.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:09.594+0000 7fe7bbe00640 1 -- 172.21.15.47:0/886026767 wait complete. 2024-04-24T22:44:10.617 INFO:tasks.ceph:Scrubbing osd.0 2024-04-24T22:44:10.618 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-24T22:44:10.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.834+0000 7f36b9200640 1 Processor -- start 2024-04-24T22:44:10.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.834+0000 7f36b9200640 1 -- start start 2024-04-24T22:44:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.836+0000 7f36b9200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f36b41526b0 0x7f36b40c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.836+0000 7f36b9200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b40c7f10 con 0x7f36b40c79e0 2024-04-24T22:44:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.836+0000 7f36b9200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b40c8190 con 0x7f36b41522d0 2024-04-24T22:44:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.836+0000 7f36b9200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f36b40c8470 con 0x7f36b41526b0 2024-04-24T22:44:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.836+0000 7f36b3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f36b41522d0 0x7f36b4158170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:45280/0 (socket says 172.21.15.47:45280) 2024-04-24T22:44:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.836+0000 7f36b3400640 1 -- 172.21.15.47:0/483436432 learned_addr learned my addr 172.21.15.47:0/483436432 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.836+0000 7f36b2a00640 1 --2- 172.21.15.47:0/483436432 >> v2:172.21.15.47:3300/0 conn(0x7f36b41526b0 0x7f36b40c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.837+0000 7f36b2000640 1 -- 172.21.15.47:0/483436432 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 906588217 0 0) 0x7f36b40c8190 con 0x7f36b41522d0 2024-04-24T22:44:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.837+0000 7f36b2000640 1 -- 172.21.15.47:0/483436432 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3698003520 con 0x7f36b41522d0 2024-04-24T22:44:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.837+0000 7f36b2a00640 1 -- 172.21.15.47:0/483436432 >> v1:172.21.15.113:6789/0 conn(0x7f36b41522d0 legacy=0x7f36b4158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.837+0000 7f36b2a00640 1 -- 172.21.15.47:0/483436432 >> v1:172.21.15.47:6789/0 conn(0x7f36b40c79e0 legacy=0x7f36b40c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.837+0000 7f36b2a00640 1 -- 172.21.15.47:0/483436432 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b40c5bd0 con 0x7f36b41526b0 2024-04-24T22:44:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.838+0000 7f36b2a00640 1 --2- 172.21.15.47:0/483436432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b41526b0 0x7f36b40c3130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36a00097a0 tx=0x7f36a002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafffde16ced8adf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.838+0000 7f36b2000640 1 -- 172.21.15.47:0/483436432 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0037070 con 0x7f36b41526b0 2024-04-24T22:44:10.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.838+0000 7f36b2000640 1 -- 172.21.15.47:0/483436432 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a0032640 con 0x7f36b41526b0 2024-04-24T22:44:10.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.838+0000 7f36b2000640 1 -- 172.21.15.47:0/483436432 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0032a10 con 0x7f36b41526b0 2024-04-24T22:44:10.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.839+0000 7f36b9200640 1 -- 172.21.15.47:0/483436432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b41526b0 msgr2=0x7f36b40c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:10.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.839+0000 7f36b9200640 1 --2- 172.21.15.47:0/483436432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b41526b0 0x7f36b40c3130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36a00097a0 tx=0x7f36a002a2c0 comp rx=0 tx=0).stop 2024-04-24T22:44:10.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.839+0000 7f36b9200640 1 -- 172.21.15.47:0/483436432 shutdown_connections 2024-04-24T22:44:10.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.839+0000 7f36b9200640 1 --2- 172.21.15.47:0/483436432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b41526b0 0x7f36b40c3130 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:10.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.839+0000 7f36b9200640 1 -- 172.21.15.47:0/483436432 >> 172.21.15.47:0/483436432 conn(0x7f36b40cbc30 msgr2=0x7f36b40cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:10.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.839+0000 7f36b9200640 1 -- 172.21.15.47:0/483436432 shutdown_connections 2024-04-24T22:44:10.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.839+0000 7f36b9200640 1 -- 172.21.15.47:0/483436432 wait complete. 2024-04-24T22:44:10.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.841+0000 7f36b9200640 1 Processor -- start 2024-04-24T22:44:10.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.841+0000 7f36b9200640 1 -- start start 2024-04-24T22:44:10.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40c79e0 0x7f36b40c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:10.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b9200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b41522d0 0x7f36b41530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b41526b0 0x7f36b4153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b9200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36b406d7e0 con 0x7f36b41526b0 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b9200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f36b406d660 con 0x7f36b40c79e0 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b9200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f36b40c4050 con 0x7f36b41522d0 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40c79e0 0x7f36b40c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40c79e0 0x7f36b40c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60224/0 (socket says 172.21.15.47:60224) 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b41526b0 0x7f36b4153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b3400640 1 -- 172.21.15.47:0/2494794832 learned_addr learned my addr 172.21.15.47:0/2494794832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.842+0000 7f36b2a00640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b41522d0 0x7f36b41530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.843+0000 7f36b3400640 1 -- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b41522d0 msgr2=0x7f36b41530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.843+0000 7f36b3400640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b41522d0 0x7f36b41530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.843+0000 7f36b3400640 1 -- 172.21.15.47:0/2494794832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b41526b0 msgr2=0x7f36b4153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.843+0000 7f36b3400640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b41526b0 0x7f36b4153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:10.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.843+0000 7f36b3400640 1 -- 172.21.15.47:0/2494794832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b4155a40 con 0x7f36b40c79e0 2024-04-24T22:44:10.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.843+0000 7f36b3400640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40c79e0 0x7f36b40c2140 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c560 tx=0x7f36a400ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:10.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.843+0000 7f36ab400640 1 -- 172.21.15.47:0/2494794832 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4017070 con 0x7f36b40c79e0 2024-04-24T22:44:10.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.843+0000 7f36ab400640 1 -- 172.21.15.47:0/2494794832 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a4005ce0 con 0x7f36b40c79e0 2024-04-24T22:44:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.843+0000 7f36ab400640 1 -- 172.21.15.47:0/2494794832 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40072b0 con 0x7f36b40c79e0 2024-04-24T22:44:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.843+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b40afb20 con 0x7f36b40c79e0 2024-04-24T22:44:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.844+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b4155760 con 0x7f36b40c79e0 2024-04-24T22:44:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.845+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36b414bf70 con 0x7f36b40c79e0 2024-04-24T22:44:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.845+0000 7f36ab400640 1 -- 172.21.15.47:0/2494794832 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f36a401a070 con 0x7f36b40c79e0 2024-04-24T22:44:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.846+0000 7f36ab400640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f369406be30 0x7f369406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.847+0000 7f36ab400640 1 -- 172.21.15.47:0/2494794832 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f36a408a2b0 con 0x7f36b40c79e0 2024-04-24T22:44:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.847+0000 7f36ab400640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] conn(0x7f3694074150 0x7f36940765d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.847+0000 7f36ab400640 1 -- 172.21.15.47:0/2494794832 --> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36a4014ec0 con 0x7f3694074150 2024-04-24T22:44:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.847+0000 7f36ab400640 1 -- 172.21.15.47:0/2494794832 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f36a4091050 con 0x7f36b40c79e0 2024-04-24T22:44:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.847+0000 7f36b2a00640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f369406be30 0x7f369406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.847+0000 7f36b3e00640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] conn(0x7f3694074150 0x7f36940765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.848+0000 7f36b3e00640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] conn(0x7f3694074150 0x7f36940765d0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.848+0000 7f36b2a00640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f369406be30 0x7f369406e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36a0034000 tx=0x7f36a002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.848+0000 7f36ab400640 1 -- 172.21.15.47:0/2494794832 <== osd.0 v2:172.21.15.47:6808/2543493744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36a4014ec0 con 0x7f3694074150 2024-04-24T22:44:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.864+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 --> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f36b4073b90 con 0x7f3694074150 2024-04-24T22:44:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.864+0000 7f36ab400640 1 -- 172.21.15.47:0/2494794832 <== osd.0 v2:172.21.15.47:6808/2543493744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f36b4073b90 con 0x7f3694074150 2024-04-24T22:44:10.867 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:44:10.867 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T22:44:10.867 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:44:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 >> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] conn(0x7f3694074150 msgr2=0x7f36940765d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b9200640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.47:6808/2543493744,v1:172.21.15.47:6809/2543493744] conn(0x7f3694074150 0x7f36940765d0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f369406be30 msgr2=0x7f369406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b9200640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f369406be30 0x7f369406e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36a0034000 tx=0x7f36a002e000 comp rx=0 tx=0).stop 2024-04-24T22:44:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40c79e0 msgr2=0x7f36b40c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b9200640 1 --2- 172.21.15.47:0/2494794832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40c79e0 0x7f36b40c2140 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c560 tx=0x7f36a400ca20 comp rx=0 tx=0).stop 2024-04-24T22:44:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b3e00640 1 -- 172.21.15.47:0/2494794832 reap_dead start 2024-04-24T22:44:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 shutdown_connections 2024-04-24T22:44:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 >> 172.21.15.47:0/2494794832 conn(0x7f36b40cbc30 msgr2=0x7f36b40c86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 shutdown_connections 2024-04-24T22:44:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:10.865+0000 7f36b9200640 1 -- 172.21.15.47:0/2494794832 wait complete. 2024-04-24T22:44:10.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-24T22:44:11.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.089+0000 7f0669c00640 1 Processor -- start 2024-04-24T22:44:11.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.089+0000 7f0669c00640 1 -- start start 2024-04-24T22:44:11.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0669c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06640ca770 0x7f066406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0669c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0669c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06641581b0 0x7f0664141110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0669c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f06640c57a0 con 0x7f06640ca770 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0669c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f06640c5c30 con 0x7f06641581b0 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0669c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f06640c4eb0 con 0x7f06640cab50 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0663e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06640ca770 0x7f066406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0663400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0669200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06641581b0 0x7f0664141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0663400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34842/0 (socket says 172.21.15.47:34842) 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.091+0000 7f0663400640 1 -- 172.21.15.47:0/3148146717 learned_addr learned my addr 172.21.15.47:0/3148146717 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.092+0000 7f0669200640 1 -- 172.21.15.47:0/3148146717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 msgr2=0x7f06640c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.092+0000 7f0669200640 1 --2- 172.21.15.47:0/3148146717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.092+0000 7f0669200640 1 -- 172.21.15.47:0/3148146717 >> v2:172.21.15.47:3300/0 conn(0x7f06640ca770 msgr2=0x7f066406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.092+0000 7f0669200640 1 --2- 172.21.15.47:0/3148146717 >> v2:172.21.15.47:3300/0 conn(0x7f06640ca770 0x7f066406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.092+0000 7f0669200640 1 -- 172.21.15.47:0/3148146717 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06640c8c70 con 0x7f06641581b0 2024-04-24T22:44:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.092+0000 7f0663400640 1 --2- 172.21.15.47:0/3148146717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.093+0000 7f0669200640 1 --2- 172.21.15.47:0/3148146717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 0x7f0664141110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0664062ca0 tx=0x7f065c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=642a04d0f49e0667 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.093+0000 7f0662a00640 1 -- 172.21.15.47:0/3148146717 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c007740 con 0x7f06641581b0 2024-04-24T22:44:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.093+0000 7f0662a00640 1 -- 172.21.15.47:0/3148146717 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065c0078e0 con 0x7f06641581b0 2024-04-24T22:44:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.093+0000 7f0662a00640 1 -- 172.21.15.47:0/3148146717 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c007cb0 con 0x7f06641581b0 2024-04-24T22:44:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.094+0000 7f0669c00640 1 -- 172.21.15.47:0/3148146717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 msgr2=0x7f0664141110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.094+0000 7f0669c00640 1 --2- 172.21.15.47:0/3148146717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 0x7f0664141110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0664062ca0 tx=0x7f065c02a660 comp rx=0 tx=0).stop 2024-04-24T22:44:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.094+0000 7f0669c00640 1 -- 172.21.15.47:0/3148146717 shutdown_connections 2024-04-24T22:44:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.094+0000 7f0669c00640 1 --2- 172.21.15.47:0/3148146717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 0x7f0664141110 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.094+0000 7f0669c00640 1 --2- 172.21.15.47:0/3148146717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.094+0000 7f0669c00640 1 --2- 172.21.15.47:0/3148146717 >> v2:172.21.15.47:3300/0 conn(0x7f06640ca770 0x7f066406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.094+0000 7f0669c00640 1 -- 172.21.15.47:0/3148146717 >> 172.21.15.47:0/3148146717 conn(0x7f0664072c40 msgr2=0x7f0664073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.094+0000 7f0669c00640 1 -- 172.21.15.47:0/3148146717 shutdown_connections 2024-04-24T22:44:11.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.094+0000 7f0669c00640 1 -- 172.21.15.47:0/3148146717 wait complete. 2024-04-24T22:44:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.096+0000 7f0669c00640 1 Processor -- start 2024-04-24T22:44:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.096+0000 7f0669c00640 1 -- start start 2024-04-24T22:44:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0669c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06640ca770 0x7f066413e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0669c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0669c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 0x7f0664156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0669c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f066406e540 con 0x7f06640ca770 2024-04-24T22:44:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0669c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06640c5430 con 0x7f06641581b0 2024-04-24T22:44:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0669c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f06640c8f50 con 0x7f06640cab50 2024-04-24T22:44:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0663e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06640ca770 0x7f066413e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0669200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 0x7f0664156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0663400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0663400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34848/0 (socket says 172.21.15.47:34848) 2024-04-24T22:44:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0669200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 0x7f0664156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60244/0 (socket says 172.21.15.47:60244) 2024-04-24T22:44:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0663e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06640ca770 0x7f066413e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35670/0 (socket says 172.21.15.47:35670) 2024-04-24T22:44:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.097+0000 7f0663400640 1 -- 172.21.15.47:0/684501043 learned_addr learned my addr 172.21.15.47:0/684501043 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:11.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f0663400640 1 -- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 msgr2=0x7f0664156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f0663400640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 0x7f0664156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f0663400640 1 -- 172.21.15.47:0/684501043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06640ca770 msgr2=0x7f066413e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f0663400640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06640ca770 0x7f066413e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f0663400640 1 -- 172.21.15.47:0/684501043 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06640cd720 con 0x7f06640cab50 2024-04-24T22:44:11.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f0669200640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 0x7f0664156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:11.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f0663400640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640cb7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f065000e9d0 tx=0x7f065000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f065be00640 1 -- 172.21.15.47:0/684501043 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650017070 con 0x7f06640cab50 2024-04-24T22:44:11.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f065be00640 1 -- 172.21.15.47:0/684501043 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065000ca60 con 0x7f06640cab50 2024-04-24T22:44:11.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f0669c00640 1 -- 172.21.15.47:0/684501043 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06640cda00 con 0x7f06640cab50 2024-04-24T22:44:11.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.098+0000 7f065be00640 1 -- 172.21.15.47:0/684501043 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065000ce10 con 0x7f06640cab50 2024-04-24T22:44:11.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.099+0000 7f0669c00640 1 -- 172.21.15.47:0/684501043 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0664156300 con 0x7f06640cab50 2024-04-24T22:44:11.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.100+0000 7f0669c00640 1 -- 172.21.15.47:0/684501043 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0664155e10 con 0x7f06640cab50 2024-04-24T22:44:11.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.101+0000 7f065be00640 1 -- 172.21.15.47:0/684501043 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f065001a070 con 0x7f06640cab50 2024-04-24T22:44:11.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.102+0000 7f065be00640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f064406be30 0x7f064406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.102+0000 7f065be00640 1 -- 172.21.15.47:0/684501043 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f065008b150 con 0x7f06640cab50 2024-04-24T22:44:11.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.102+0000 7f0663e00640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f064406be30 0x7f064406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.103+0000 7f0663e00640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f064406be30 0x7f064406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0654009b40 tx=0x7f0654009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.106+0000 7f065be00640 1 -- 172.21.15.47:0/684501043 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0650059390 con 0x7f06640cab50 2024-04-24T22:44:11.155 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:44:11.162 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:44:11.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.310+0000 7f0669c00640 1 -- 172.21.15.47:0/684501043 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f06640c4800 con 0x7f064406be30 2024-04-24T22:44:11.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.311+0000 7f065be00640 1 -- 172.21.15.47:0/684501043 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f06640c4800 con 0x7f064406be30 2024-04-24T22:44:11.314 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 0 to deep-scrub 2024-04-24T22:44:11.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 -- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f064406be30 msgr2=0x7f064406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f064406be30 0x7f064406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0654009b40 tx=0x7f0654009190 comp rx=0 tx=0).stop 2024-04-24T22:44:11.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 -- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 msgr2=0x7f06640cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640cb7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f065000e9d0 tx=0x7f065000ee90 comp rx=0 tx=0).stop 2024-04-24T22:44:11.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 -- 172.21.15.47:0/684501043 shutdown_connections 2024-04-24T22:44:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f064406be30 0x7f064406e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06641581b0 0x7f0664156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06640cab50 0x7f06640cb7b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 --2- 172.21.15.47:0/684501043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06640ca770 0x7f066413e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 -- 172.21.15.47:0/684501043 >> 172.21.15.47:0/684501043 conn(0x7f0664072c40 msgr2=0x7f0664153610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.314+0000 7f0669c00640 1 -- 172.21.15.47:0/684501043 shutdown_connections 2024-04-24T22:44:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.315+0000 7f0669c00640 1 -- 172.21.15.47:0/684501043 wait complete. 2024-04-24T22:44:11.337 INFO:tasks.ceph:Scrubbing osd.1 2024-04-24T22:44:11.337 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-24T22:44:11.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.551+0000 7fb2e5800640 1 Processor -- start 2024-04-24T22:44:11.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.551+0000 7fb2e5800640 1 -- start start 2024-04-24T22:44:11.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.553+0000 7fb2e5800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e00b1190 0x7fb2e00c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.553+0000 7fb2e5800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2e00c0b90 0x7fb2e00c7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.553+0000 7fb2e5800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e00c6790 con 0x7fb2e0154660 2024-04-24T22:44:11.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.553+0000 7fb2e5800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e00c6cf0 con 0x7fb2e00c0b90 2024-04-24T22:44:11.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.553+0000 7fb2e5800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e006d2c0 con 0x7fb2e00b1190 2024-04-24T22:44:11.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.553+0000 7fb2e4e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2e00c0b90 0x7fb2e00c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.553+0000 7fb2e4e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb2e00c0b90 0x7fb2e00c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35684/0 (socket says 172.21.15.47:35684) 2024-04-24T22:44:11.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.553+0000 7fb2e4e00640 1 -- 172.21.15.47:0/2113875826 learned_addr learned my addr 172.21.15.47:0/2113875826 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:11.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.554+0000 7fb2df400640 1 --2- 172.21.15.47:0/2113875826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e00b1190 0x7fb2e00c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.554+0000 7fb2e4e00640 1 -- 172.21.15.47:0/2113875826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e00b1190 msgr2=0x7fb2e00c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.554+0000 7fb2e4e00640 1 --2- 172.21.15.47:0/2113875826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e00b1190 0x7fb2e00c1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.554+0000 7fb2e4e00640 1 -- 172.21.15.47:0/2113875826 >> v1:172.21.15.47:6789/0 conn(0x7fb2e0154660 legacy=0x7fb2e015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.554+0000 7fb2e4e00640 1 -- 172.21.15.47:0/2113875826 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e00b0dc0 con 0x7fb2e00c0b90 2024-04-24T22:44:11.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.555+0000 7fb2e4e00640 1 --2- 172.21.15.47:0/2113875826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00c0b90 0x7fb2e00c7610 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009870 tx=0x7fb2d802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7daab914b41c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.555+0000 7fb2dea00640 1 -- 172.21.15.47:0/2113875826 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8037070 con 0x7fb2e00c0b90 2024-04-24T22:44:11.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.555+0000 7fb2dea00640 1 -- 172.21.15.47:0/2113875826 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d8032440 con 0x7fb2e00c0b90 2024-04-24T22:44:11.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.555+0000 7fb2dea00640 1 -- 172.21.15.47:0/2113875826 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8032810 con 0x7fb2e00c0b90 2024-04-24T22:44:11.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.556+0000 7fb2e5800640 1 -- 172.21.15.47:0/2113875826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00c0b90 msgr2=0x7fb2e00c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.556+0000 7fb2e5800640 1 --2- 172.21.15.47:0/2113875826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00c0b90 0x7fb2e00c7610 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009870 tx=0x7fb2d802a1c0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.556+0000 7fb2e5800640 1 -- 172.21.15.47:0/2113875826 shutdown_connections 2024-04-24T22:44:11.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.556+0000 7fb2e5800640 1 --2- 172.21.15.47:0/2113875826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00c0b90 0x7fb2e00c7610 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.556+0000 7fb2e5800640 1 --2- 172.21.15.47:0/2113875826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e00b1190 0x7fb2e00c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.556+0000 7fb2e5800640 1 -- 172.21.15.47:0/2113875826 >> 172.21.15.47:0/2113875826 conn(0x7fb2e00cbc30 msgr2=0x7fb2e00cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.556+0000 7fb2e5800640 1 -- 172.21.15.47:0/2113875826 shutdown_connections 2024-04-24T22:44:11.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.556+0000 7fb2e5800640 1 -- 172.21.15.47:0/2113875826 wait complete. 2024-04-24T22:44:11.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.558+0000 7fb2e5800640 1 Processor -- start 2024-04-24T22:44:11.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.558+0000 7fb2e5800640 1 -- start start 2024-04-24T22:44:11.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.559+0000 7fb2e5800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00b1190 0x7fb2e01f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.559+0000 7fb2e5800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e00c0b90 0x7fb2e014f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.559+0000 7fb2e5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e0154660 0x7fb2e01469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.559+0000 7fb2e5800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e00c1940 con 0x7fb2e00b1190 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.559+0000 7fb2e5800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e00c1660 con 0x7fb2e0154660 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.559+0000 7fb2e5800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e0154a40 con 0x7fb2e00c0b90 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.559+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00b1190 0x7fb2e01f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00b1190 0x7fb2e01f40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35690/0 (socket says 172.21.15.47:35690) 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2dfe00640 1 -- 172.21.15.47:0/2686377855 learned_addr learned my addr 172.21.15.47:0/2686377855 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2e4e00640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e0154660 0x7fb2e01469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2df400640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e00c0b90 0x7fb2e014f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2dfe00640 1 -- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e00c0b90 msgr2=0x7fb2e014f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2dfe00640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2e00c0b90 0x7fb2e014f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2dfe00640 1 -- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e0154660 msgr2=0x7fb2e01469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2dfe00640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e0154660 0x7fb2e01469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2dfe00640 1 -- 172.21.15.47:0/2686377855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e00716c0 con 0x7fb2e00b1190 2024-04-24T22:44:11.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2e4e00640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2e0154660 0x7fb2e01469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:11.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.560+0000 7fb2dfe00640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00b1190 0x7fb2e01f40f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c730 tx=0x7fb2d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.561+0000 7fb2d7e00640 1 -- 172.21.15.47:0/2686377855 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0016020 con 0x7fb2e00b1190 2024-04-24T22:44:11.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.561+0000 7fb2d7e00640 1 -- 172.21.15.47:0/2686377855 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d0006d50 con 0x7fb2e00b1190 2024-04-24T22:44:11.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.561+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e01f0170 con 0x7fb2e00b1190 2024-04-24T22:44:11.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.561+0000 7fb2d7e00640 1 -- 172.21.15.47:0/2686377855 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0003bb0 con 0x7fb2e00b1190 2024-04-24T22:44:11.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.561+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e01f0710 con 0x7fb2e00b1190 2024-04-24T22:44:11.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.562+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb2e0073b90 con 0x7fb2e00b1190 2024-04-24T22:44:11.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.563+0000 7fb2d7e00640 1 -- 172.21.15.47:0/2686377855 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2d000a850 con 0x7fb2e00b1190 2024-04-24T22:44:11.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.564+0000 7fb2d7e00640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 0x7fb2c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.564+0000 7fb2d7e00640 1 -- 172.21.15.47:0/2686377855 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb2d008aa10 con 0x7fb2e00b1190 2024-04-24T22:44:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.564+0000 7fb2d7e00640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] conn(0x7fb2c00741b0 0x7fb2c0076630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.564+0000 7fb2d7e00640 1 -- 172.21.15.47:0/2686377855 --> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2d001eec0 con 0x7fb2c00741b0 2024-04-24T22:44:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.564+0000 7fb2d7e00640 1 -- 172.21.15.47:0/2686377855 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb2d008f2c0 con 0x7fb2e00b1190 2024-04-24T22:44:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.564+0000 7fb2df400640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 0x7fb2c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.565+0000 7fb2e4e00640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] conn(0x7fb2c00741b0 0x7fb2c0076630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.565+0000 7fb2df400640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 0x7fb2c006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0150180 tx=0x7fb2cc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.565+0000 7fb2e4e00640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] conn(0x7fb2c00741b0 0x7fb2c0076630 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.566+0000 7fb2d7e00640 1 -- 172.21.15.47:0/2686377855 <== osd.1 v2:172.21.15.47:6800/623040628 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2d001eec0 con 0x7fb2c00741b0 2024-04-24T22:44:11.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.581+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 --> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb2e00c71e0 con 0x7fb2c00741b0 2024-04-24T22:44:11.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.581+0000 7fb2d7e00640 1 -- 172.21.15.47:0/2686377855 <== osd.1 v2:172.21.15.47:6800/623040628 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb2e00c71e0 con 0x7fb2c00741b0 2024-04-24T22:44:11.584 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:44:11.584 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T22:44:11.584 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:44:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 >> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] conn(0x7fb2c00741b0 msgr2=0x7fb2c0076630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e5800640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.47:6800/623040628,v1:172.21.15.47:6801/623040628] conn(0x7fb2c00741b0 0x7fb2c0076630 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 msgr2=0x7fb2c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e5800640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb2c006be30 0x7fb2c006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0150180 tx=0x7fb2cc009190 comp rx=0 tx=0).stop 2024-04-24T22:44:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00b1190 msgr2=0x7fb2e01f40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e5800640 1 --2- 172.21.15.47:0/2686377855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2e00b1190 0x7fb2e01f40f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c730 tx=0x7fb2d000cbf0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e4e00640 1 -- 172.21.15.47:0/2686377855 reap_dead start 2024-04-24T22:44:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 shutdown_connections 2024-04-24T22:44:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 >> 172.21.15.47:0/2686377855 conn(0x7fb2e00cbc30 msgr2=0x7fb2e00cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 shutdown_connections 2024-04-24T22:44:11.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.582+0000 7fb2e5800640 1 -- 172.21.15.47:0/2686377855 wait complete. 2024-04-24T22:44:11.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-24T22:44:11.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.816+0000 7fea51400640 1 Processor -- start 2024-04-24T22:44:11.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.816+0000 7fea51400640 1 -- start start 2024-04-24T22:44:11.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.817+0000 7fea51400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea4c0cc840 0x7fea4c0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.817+0000 7fea51400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c0cbde0 0x7fea4c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.817+0000 7fea51400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea4c15b240 con 0x7fea4c073380 2024-04-24T22:44:11.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.817+0000 7fea51400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fea4c15daa0 con 0x7fea4c0cc840 2024-04-24T22:44:11.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.817+0000 7fea51400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fea4c15dcf0 con 0x7fea4c0cbde0 2024-04-24T22:44:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.818+0000 7fea41e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fea4c073380 0x7fea4c0cb6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46628/0 (socket says 172.21.15.47:46628) 2024-04-24T22:44:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.818+0000 7fea41e00640 1 -- 172.21.15.47:0/1165834445 learned_addr learned my addr 172.21.15.47:0/1165834445 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.818+0000 7fea4b200640 1 --2- 172.21.15.47:0/1165834445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c0cbde0 0x7fea4c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.818+0000 7fea4a800640 1 --2- 172.21.15.47:0/1165834445 >> v2:172.21.15.113:3300/0 conn(0x7fea4c0cc840 0x7fea4c0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.818+0000 7fea4a800640 1 -- 172.21.15.47:0/1165834445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c0cbde0 msgr2=0x7fea4c0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.818+0000 7fea4a800640 1 --2- 172.21.15.47:0/1165834445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c0cbde0 0x7fea4c0ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.818+0000 7fea4a800640 1 -- 172.21.15.47:0/1165834445 >> v1:172.21.15.47:6789/0 conn(0x7fea4c073380 legacy=0x7fea4c0cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.818+0000 7fea49e00640 1 -- 172.21.15.47:0/1165834445 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719162319 0 0) 0x7fea4c15b240 con 0x7fea4c073380 2024-04-24T22:44:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.818+0000 7fea4a800640 1 -- 172.21.15.47:0/1165834445 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c15dfd0 con 0x7fea4c0cc840 2024-04-24T22:44:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.819+0000 7fea4a800640 1 --2- 172.21.15.47:0/1165834445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cc840 0x7fea4c0ca7d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0097a0 tx=0x7fea3c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=308bfeb62bcf77b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.820+0000 7fea49e00640 1 -- 172.21.15.47:0/1165834445 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c037070 con 0x7fea4c0cc840 2024-04-24T22:44:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.820+0000 7fea49e00640 1 -- 172.21.15.47:0/1165834445 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3c032440 con 0x7fea4c0cc840 2024-04-24T22:44:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.820+0000 7fea49e00640 1 -- 172.21.15.47:0/1165834445 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c032810 con 0x7fea4c0cc840 2024-04-24T22:44:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.820+0000 7fea51400640 1 -- 172.21.15.47:0/1165834445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cc840 msgr2=0x7fea4c0ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.820+0000 7fea51400640 1 --2- 172.21.15.47:0/1165834445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cc840 0x7fea4c0ca7d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0097a0 tx=0x7fea3c02a180 comp rx=0 tx=0).stop 2024-04-24T22:44:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.820+0000 7fea51400640 1 -- 172.21.15.47:0/1165834445 shutdown_connections 2024-04-24T22:44:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.820+0000 7fea51400640 1 --2- 172.21.15.47:0/1165834445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c0cbde0 0x7fea4c0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.820+0000 7fea51400640 1 --2- 172.21.15.47:0/1165834445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cc840 0x7fea4c0ca7d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.820+0000 7fea51400640 1 -- 172.21.15.47:0/1165834445 >> 172.21.15.47:0/1165834445 conn(0x7fea4c06c940 msgr2=0x7fea4c072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.821+0000 7fea51400640 1 -- 172.21.15.47:0/1165834445 shutdown_connections 2024-04-24T22:44:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.821+0000 7fea51400640 1 -- 172.21.15.47:0/1165834445 wait complete. 2024-04-24T22:44:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.822+0000 7fea51400640 1 Processor -- start 2024-04-24T22:44:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.822+0000 7fea51400640 1 -- start start 2024-04-24T22:44:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.823+0000 7fea51400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c073380 0x7fea4c1f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.823+0000 7fea51400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cbde0 0x7fea4c13eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.823+0000 7fea51400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0cc840 0x7fea4c0b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.823+0000 7fea51400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c15e870 con 0x7fea4c0cc840 2024-04-24T22:44:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.823+0000 7fea51400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c15e590 con 0x7fea4c0cbde0 2024-04-24T22:44:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.823+0000 7fea51400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fea4c15af30 con 0x7fea4c073380 2024-04-24T22:44:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea4b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0cc840 0x7fea4c0b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea4a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c073380 0x7fea4c1f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea41e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cbde0 0x7fea4c13eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea4a800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c073380 0x7fea4c1f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34900/0 (socket says 172.21.15.47:34900) 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea4a800640 1 -- 172.21.15.47:0/228090499 learned_addr learned my addr 172.21.15.47:0/228090499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea41e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cbde0 0x7fea4c13eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60282/0 (socket says 172.21.15.47:60282) 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea4b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0cc840 0x7fea4c0b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35696/0 (socket says 172.21.15.47:35696) 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea41e00640 1 -- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c073380 msgr2=0x7fea4c1f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea41e00640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c073380 0x7fea4c1f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea41e00640 1 -- 172.21.15.47:0/228090499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0cc840 msgr2=0x7fea4c0b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea41e00640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0cc840 0x7fea4c0b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea41e00640 1 -- 172.21.15.47:0/228090499 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c0b4b20 con 0x7fea4c0cbde0 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea4a800640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c073380 0x7fea4c1f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea4b200640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0cc840 0x7fea4c0b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.824+0000 7fea41e00640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cbde0 0x7fea4c13eae0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea3400c410 tx=0x7fea3400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.825+0000 7fea43400640 1 -- 172.21.15.47:0/228090499 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340074e0 con 0x7fea4c0cbde0 2024-04-24T22:44:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.825+0000 7fea43400640 1 -- 172.21.15.47:0/228090499 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea340047e0 con 0x7fea4c0cbde0 2024-04-24T22:44:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.825+0000 7fea43400640 1 -- 172.21.15.47:0/228090499 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34004bd0 con 0x7fea4c0cbde0 2024-04-24T22:44:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.825+0000 7fea51400640 1 -- 172.21.15.47:0/228090499 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c0b4dc0 con 0x7fea4c0cbde0 2024-04-24T22:44:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.825+0000 7fea51400640 1 -- 172.21.15.47:0/228090499 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c0b34f0 con 0x7fea4c0cbde0 2024-04-24T22:44:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.826+0000 7fea51400640 1 -- 172.21.15.47:0/228090499 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4c073b90 con 0x7fea4c0cbde0 2024-04-24T22:44:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.827+0000 7fea43400640 1 -- 172.21.15.47:0/228090499 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea34011070 con 0x7fea4c0cbde0 2024-04-24T22:44:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.828+0000 7fea43400640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea2806be30 0x7fea2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.828+0000 7fea43400640 1 -- 172.21.15.47:0/228090499 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fea3408a970 con 0x7fea4c0cbde0 2024-04-24T22:44:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.828+0000 7fea4a800640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea2806be30 0x7fea2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.829+0000 7fea4a800640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea2806be30 0x7fea2806e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea3c02a1c0 tx=0x7fea3c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:11.833+0000 7fea43400640 1 -- 172.21.15.47:0/228090499 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fea34058c70 con 0x7fea4c0cbde0 2024-04-24T22:44:12.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.042+0000 7fea51400640 1 -- 172.21.15.47:0/228090499 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fea4c153d10 con 0x7fea2806be30 2024-04-24T22:44:12.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.043+0000 7fea43400640 1 -- 172.21.15.47:0/228090499 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fea4c153d10 con 0x7fea2806be30 2024-04-24T22:44:12.046 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 1 to deep-scrub 2024-04-24T22:44:12.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.046+0000 7fea51400640 1 -- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea2806be30 msgr2=0x7fea2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.046+0000 7fea51400640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea2806be30 0x7fea2806e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea3c02a1c0 tx=0x7fea3c02e000 comp rx=0 tx=0).stop 2024-04-24T22:44:12.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.046+0000 7fea51400640 1 -- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cbde0 msgr2=0x7fea4c13eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.046+0000 7fea51400640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cbde0 0x7fea4c13eae0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea3400c410 tx=0x7fea3400c8d0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.047+0000 7fea51400640 1 -- 172.21.15.47:0/228090499 shutdown_connections 2024-04-24T22:44:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.047+0000 7fea51400640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fea2806be30 0x7fea2806e2f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.047+0000 7fea51400640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c0cc840 0x7fea4c0b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.047+0000 7fea51400640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4c0cbde0 0x7fea4c13eae0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.047+0000 7fea51400640 1 --2- 172.21.15.47:0/228090499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea4c073380 0x7fea4c1f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.047+0000 7fea51400640 1 -- 172.21.15.47:0/228090499 >> 172.21.15.47:0/228090499 conn(0x7fea4c06c940 msgr2=0x7fea4c150710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.047+0000 7fea51400640 1 -- 172.21.15.47:0/228090499 shutdown_connections 2024-04-24T22:44:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.047+0000 7fea51400640 1 -- 172.21.15.47:0/228090499 wait complete. 2024-04-24T22:44:12.068 INFO:tasks.ceph:Scrubbing osd.2 2024-04-24T22:44:12.069 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-24T22:44:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.289+0000 7f96ce200640 1 Processor -- start 2024-04-24T22:44:12.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.289+0000 7f96ce200640 1 -- start start 2024-04-24T22:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.290+0000 7f96ce200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c80b2ea0 0x7f96c80cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.291+0000 7f96ce200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c8154bd0 con 0x7f96c80cc110 2024-04-24T22:44:12.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.291+0000 7f96ce200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c815dbd0 con 0x7f96c81547f0 2024-04-24T22:44:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.291+0000 7f96ce200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f96c815deb0 con 0x7f96c80b2ea0 2024-04-24T22:44:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.291+0000 7f96cce00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f96c81547f0 0x7f96c80cad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:45292/0 (socket says 172.21.15.47:45292) 2024-04-24T22:44:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.291+0000 7f96cce00640 1 -- 172.21.15.47:0/209017484 learned_addr learned my addr 172.21.15.47:0/209017484 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:12.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.291+0000 7f96c7e00640 1 --2- 172.21.15.47:0/209017484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c80b2ea0 0x7f96c80cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.291+0000 7f96c7400640 1 -- 172.21.15.47:0/209017484 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 430124191 0 0) 0x7f96c815dbd0 con 0x7f96c81547f0 2024-04-24T22:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.292+0000 7f96c7400640 1 -- 172.21.15.47:0/209017484 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96ac003520 con 0x7f96c81547f0 2024-04-24T22:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.292+0000 7f96c7400640 1 -- 172.21.15.47:0/209017484 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 261466207 0 0) 0x7f96c8154bd0 con 0x7f96c80cc110 2024-04-24T22:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.292+0000 7f96c7400640 1 -- 172.21.15.47:0/209017484 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c815dbd0 con 0x7f96c80cc110 2024-04-24T22:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.292+0000 7f96c7e00640 1 -- 172.21.15.47:0/209017484 >> v1:172.21.15.113:6789/0 conn(0x7f96c81547f0 legacy=0x7f96c80cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.292+0000 7f96c7e00640 1 -- 172.21.15.47:0/209017484 >> v1:172.21.15.47:6789/0 conn(0x7f96c80cc110 legacy=0x7f96c815a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.292+0000 7f96c7e00640 1 -- 172.21.15.47:0/209017484 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c815e190 con 0x7f96c80b2ea0 2024-04-24T22:44:12.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.293+0000 7f96c7e00640 1 --2- 172.21.15.47:0/209017484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c80b2ea0 0x7f96c80cfd10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96b80097d0 tx=0x7f96b802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ebfff7ce376412 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:12.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.293+0000 7f96c7400640 1 -- 172.21.15.47:0/209017484 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8037070 con 0x7f96c80b2ea0 2024-04-24T22:44:12.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.293+0000 7f96c7400640 1 -- 172.21.15.47:0/209017484 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b802ae50 con 0x7f96c80b2ea0 2024-04-24T22:44:12.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.293+0000 7f96c7400640 1 -- 172.21.15.47:0/209017484 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8032660 con 0x7f96c80b2ea0 2024-04-24T22:44:12.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.294+0000 7f96ce200640 1 -- 172.21.15.47:0/209017484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c80b2ea0 msgr2=0x7f96c80cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.294+0000 7f96ce200640 1 --2- 172.21.15.47:0/209017484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c80b2ea0 0x7f96c80cfd10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96b80097d0 tx=0x7f96b802a100 comp rx=0 tx=0).stop 2024-04-24T22:44:12.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.294+0000 7f96ce200640 1 -- 172.21.15.47:0/209017484 shutdown_connections 2024-04-24T22:44:12.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.294+0000 7f96ce200640 1 --2- 172.21.15.47:0/209017484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c80b2ea0 0x7f96c80cfd10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.294+0000 7f96ce200640 1 -- 172.21.15.47:0/209017484 >> 172.21.15.47:0/209017484 conn(0x7f96c8072750 msgr2=0x7f96c8072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:12.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.294+0000 7f96ce200640 1 -- 172.21.15.47:0/209017484 shutdown_connections 2024-04-24T22:44:12.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.294+0000 7f96ce200640 1 -- 172.21.15.47:0/209017484 wait complete. 2024-04-24T22:44:12.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.296+0000 7f96ce200640 1 Processor -- start 2024-04-24T22:44:12.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.296+0000 7f96ce200640 1 -- start start 2024-04-24T22:44:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96ce200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c80b2ea0 0x7f96c814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96ce200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c80cc110 0x7f96c814ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96ce200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c81547f0 0x7f96c8158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96ce200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f96c815afa0 con 0x7f96c80cc110 2024-04-24T22:44:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96ce200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f96c815e750 con 0x7f96c81547f0 2024-04-24T22:44:12.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96ce200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f96c815b120 con 0x7f96c80b2ea0 2024-04-24T22:44:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c80cc110 0x7f96c814ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c80cc110 0x7f96c814ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35710/0 (socket says 172.21.15.47:35710) 2024-04-24T22:44:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96c7e00640 1 -- 172.21.15.47:0/3932052349 learned_addr learned my addr 172.21.15.47:0/3932052349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96cce00640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c80b2ea0 0x7f96c814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.297+0000 7f96cd800640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c81547f0 0x7f96c8158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.298+0000 7f96c7e00640 1 -- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c80b2ea0 msgr2=0x7f96c814a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.298+0000 7f96c7e00640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c80b2ea0 0x7f96c814a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.298+0000 7f96c7e00640 1 -- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c81547f0 msgr2=0x7f96c8158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.298+0000 7f96c7e00640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c81547f0 0x7f96c8158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.298+0000 7f96c7e00640 1 -- 172.21.15.47:0/3932052349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c80cfdd0 con 0x7f96c80cc110 2024-04-24T22:44:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.298+0000 7f96c7e00640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c80cc110 0x7f96c814ac40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f96b8035040 tx=0x7f96b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.298+0000 7f96c4c00640 1 -- 172.21.15.47:0/3932052349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8041070 con 0x7f96c80cc110 2024-04-24T22:44:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.298+0000 7f96c4c00640 1 -- 172.21.15.47:0/3932052349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b803c460 con 0x7f96c80cc110 2024-04-24T22:44:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.298+0000 7f96c4c00640 1 -- 172.21.15.47:0/3932052349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8037040 con 0x7f96c80cc110 2024-04-24T22:44:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.298+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c80cfaf0 con 0x7f96c80cc110 2024-04-24T22:44:12.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.299+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c80cf360 con 0x7f96c80cc110 2024-04-24T22:44:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.300+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96c80cf8a0 con 0x7f96c80cc110 2024-04-24T22:44:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.301+0000 7f96c4c00640 1 -- 172.21.15.47:0/3932052349 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96b8044070 con 0x7f96c80cc110 2024-04-24T22:44:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.302+0000 7f96c4c00640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f96a406be60 0x7f96a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.302+0000 7f96c4c00640 1 -- 172.21.15.47:0/3932052349 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f96b80ab360 con 0x7f96c80cc110 2024-04-24T22:44:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.302+0000 7f96cce00640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f96a406be60 0x7f96a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.302+0000 7f96c4c00640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] conn(0x7f96a4074180 0x7f96a4076600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.302+0000 7f96c4c00640 1 -- 172.21.15.47:0/3932052349 --> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96b8048e50 con 0x7f96a4074180 2024-04-24T22:44:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.302+0000 7f96c4c00640 1 -- 172.21.15.47:0/3932052349 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f96b80b2050 con 0x7f96c80cc110 2024-04-24T22:44:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.303+0000 7f96cd800640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] conn(0x7f96a4074180 0x7f96a4076600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.303+0000 7f96cce00640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f96a406be60 0x7f96a406e320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96b0000f60 tx=0x7f96b0007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.304+0000 7f96cd800640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] conn(0x7f96a4074180 0x7f96a4076600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:12.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.305+0000 7f96c4c00640 1 -- 172.21.15.47:0/3932052349 <== osd.2 v2:172.21.15.47:6816/4187599812 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96b8048e50 con 0x7f96a4074180 2024-04-24T22:44:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.320+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 --> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f96c8158750 con 0x7f96a4074180 2024-04-24T22:44:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.321+0000 7f96c4c00640 1 -- 172.21.15.47:0/3932052349 <== osd.2 v2:172.21.15.47:6816/4187599812 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f96c8158750 con 0x7f96a4074180 2024-04-24T22:44:12.323 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:44:12.323 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T22:44:12.324 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:44:12.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.321+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 >> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] conn(0x7f96a4074180 msgr2=0x7f96a4076600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.321+0000 7f96ce200640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.47:6816/4187599812,v1:172.21.15.47:6817/4187599812] conn(0x7f96a4074180 0x7f96a4076600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.321+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f96a406be60 msgr2=0x7f96a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.321+0000 7f96ce200640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f96a406be60 0x7f96a406e320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96b0000f60 tx=0x7f96b0007d70 comp rx=0 tx=0).stop 2024-04-24T22:44:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.321+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c80cc110 msgr2=0x7f96c814ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.321+0000 7f96ce200640 1 --2- 172.21.15.47:0/3932052349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96c80cc110 0x7f96c814ac40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f96b8035040 tx=0x7f96b80046c0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.321+0000 7f96cd800640 1 -- 172.21.15.47:0/3932052349 reap_dead start 2024-04-24T22:44:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.321+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 shutdown_connections 2024-04-24T22:44:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.321+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 >> 172.21.15.47:0/3932052349 conn(0x7f96c8072750 msgr2=0x7f96c8152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.322+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 shutdown_connections 2024-04-24T22:44:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.322+0000 7f96ce200640 1 -- 172.21.15.47:0/3932052349 wait complete. 2024-04-24T22:44:12.346 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-24T22:44:12.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.537+0000 7f18b5800640 1 Processor -- start 2024-04-24T22:44:12.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.538+0000 7f18b5800640 1 -- start start 2024-04-24T22:44:12.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.539+0000 7f18b5800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18b015dbe0 0x7f18b006b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.539+0000 7f18b5800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b015e290 con 0x7f18b00c9e50 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.539+0000 7f18b5800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b015e570 con 0x7f18b015bae0 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.539+0000 7f18b5800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f18b006c690 con 0x7f18b015dbe0 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.540+0000 7f18afe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f18b00c9e50 0x7f18b015b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46658/0 (socket says 172.21.15.47:46658) 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.540+0000 7f18afe00640 1 -- 172.21.15.47:0/1237220487 learned_addr learned my addr 172.21.15.47:0/1237220487 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.540+0000 7f18b4e00640 1 --2- 172.21.15.47:0/1237220487 >> v2:172.21.15.113:3300/0 conn(0x7f18b015dbe0 0x7f18b006b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.540+0000 7f18b4e00640 1 -- 172.21.15.47:0/1237220487 >> v1:172.21.15.113:6789/0 conn(0x7f18b015bae0 legacy=0x7f18b006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.540+0000 7f18b4e00640 1 -- 172.21.15.47:0/1237220487 >> v1:172.21.15.47:6789/0 conn(0x7f18b00c9e50 legacy=0x7f18b015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.540+0000 7f18aea00640 1 -- 172.21.15.47:0/1237220487 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 163003411 0 0) 0x7f18b015e570 con 0x7f18b015bae0 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.540+0000 7f18b4e00640 1 -- 172.21.15.47:0/1237220487 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b015e850 con 0x7f18b015dbe0 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.541+0000 7f18b4e00640 1 --2- 172.21.15.47:0/1237220487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b015dbe0 0x7f18b006b160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f18a80097a0 tx=0x7f18a802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db981e01d1572451 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.542+0000 7f18aea00640 1 -- 172.21.15.47:0/1237220487 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8037070 con 0x7f18b015dbe0 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.542+0000 7f18aea00640 1 -- 172.21.15.47:0/1237220487 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a802ae50 con 0x7f18b015dbe0 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.542+0000 7f18aea00640 1 -- 172.21.15.47:0/1237220487 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8032660 con 0x7f18b015dbe0 2024-04-24T22:44:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.542+0000 7f18b5800640 1 -- 172.21.15.47:0/1237220487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b015dbe0 msgr2=0x7f18b006b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.542+0000 7f18b5800640 1 --2- 172.21.15.47:0/1237220487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b015dbe0 0x7f18b006b160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f18a80097a0 tx=0x7f18a802a1c0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.542+0000 7f18b5800640 1 -- 172.21.15.47:0/1237220487 shutdown_connections 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.542+0000 7f18b5800640 1 --2- 172.21.15.47:0/1237220487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b015dbe0 0x7f18b006b160 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.542+0000 7f18b5800640 1 -- 172.21.15.47:0/1237220487 >> 172.21.15.47:0/1237220487 conn(0x7f18b00cfd00 msgr2=0x7f18b00d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.542+0000 7f18b5800640 1 -- 172.21.15.47:0/1237220487 shutdown_connections 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.542+0000 7f18b5800640 1 -- 172.21.15.47:0/1237220487 wait complete. 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.544+0000 7f18b5800640 1 Processor -- start 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.544+0000 7f18b5800640 1 -- start start 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.545+0000 7f18b5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b00c9e50 0x7f18b0157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.545+0000 7f18b5800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b015bae0 0x7f18b01e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.545+0000 7f18b5800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b015dbe0 0x7f18b0153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.545+0000 7f18b5800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f18b00cc5d0 con 0x7f18b015dbe0 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.545+0000 7f18b5800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18b015eb30 con 0x7f18b00c9e50 2024-04-24T22:44:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.545+0000 7f18b5800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f18b015ab60 con 0x7f18b015bae0 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.545+0000 7f18af400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b015bae0 0x7f18b01e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.545+0000 7f18b4e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b015dbe0 0x7f18b0153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.546+0000 7f18af400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b015bae0 0x7f18b01e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34934/0 (socket says 172.21.15.47:34934) 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.546+0000 7f18af400640 1 -- 172.21.15.47:0/1572226902 learned_addr learned my addr 172.21.15.47:0/1572226902 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.546+0000 7f18afe00640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b00c9e50 0x7f18b0157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.546+0000 7f18b4e00640 1 -- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b015bae0 msgr2=0x7f18b01e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.546+0000 7f18b4e00640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b015bae0 0x7f18b01e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.546+0000 7f18b4e00640 1 -- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b00c9e50 msgr2=0x7f18b0157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.546+0000 7f18b4e00640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b00c9e50 0x7f18b0157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.546+0000 7f18b4e00640 1 -- 172.21.15.47:0/1572226902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b0156540 con 0x7f18b015dbe0 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.546+0000 7f18af400640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b015bae0 0x7f18b01e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.546+0000 7f18b4e00640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b015dbe0 0x7f18b0153b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f18a8031000 tx=0x7f18a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.547+0000 7f18a7e00640 1 -- 172.21.15.47:0/1572226902 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8037040 con 0x7f18b015dbe0 2024-04-24T22:44:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.547+0000 7f18a7e00640 1 -- 172.21.15.47:0/1572226902 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a8004030 con 0x7f18b015dbe0 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.547+0000 7f18a7e00640 1 -- 172.21.15.47:0/1572226902 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8004440 con 0x7f18b015dbe0 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.547+0000 7f18b5800640 1 -- 172.21.15.47:0/1572226902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b0158070 con 0x7f18b015dbe0 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.547+0000 7f18b5800640 1 -- 172.21.15.47:0/1572226902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b0155f80 con 0x7f18b015dbe0 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.548+0000 7f18b5800640 1 -- 172.21.15.47:0/1572226902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b004f5e0 con 0x7f18b015dbe0 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.549+0000 7f18a7e00640 1 -- 172.21.15.47:0/1572226902 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18a803a070 con 0x7f18b015dbe0 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.551+0000 7f18a7e00640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f188c06be30 0x7f188c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.551+0000 7f18a7e00640 1 -- 172.21.15.47:0/1572226902 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f18a80ab350 con 0x7f18b015dbe0 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.551+0000 7f18afe00640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f188c06be30 0x7f188c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.552+0000 7f18afe00640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f188c06be30 0x7f188c06e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f18a00012e0 tx=0x7f18a0001850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.556+0000 7f18a7e00640 1 -- 172.21.15.47:0/1572226902 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f18b0156540 con 0x7f18b015dbe0 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.763+0000 7f18b5800640 1 -- 172.21.15.47:0/1572226902 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f18b00cb760 con 0x7f188c06be30 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.764+0000 7f18a7e00640 1 -- 172.21.15.47:0/1572226902 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f18b00cb760 con 0x7f188c06be30 2024-04-24T22:44:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 2 to deep-scrub 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.767+0000 7f18b5800640 1 -- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f188c06be30 msgr2=0x7f188c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.767+0000 7f18b5800640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f188c06be30 0x7f188c06e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f18a00012e0 tx=0x7f18a0001850 comp rx=0 tx=0).stop 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.768+0000 7f18b5800640 1 -- 172.21.15.47:0/1572226902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b015dbe0 msgr2=0x7f18b0153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.768+0000 7f18b5800640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b015dbe0 0x7f18b0153b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f18a8031000 tx=0x7f18a80046c0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.768+0000 7f18b5800640 1 -- 172.21.15.47:0/1572226902 shutdown_connections 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.768+0000 7f18b5800640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f188c06be30 0x7f188c06e2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.768+0000 7f18b5800640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18b015dbe0 0x7f18b0153b00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.768+0000 7f18b5800640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b015bae0 0x7f18b01e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.768+0000 7f18b5800640 1 --2- 172.21.15.47:0/1572226902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b00c9e50 0x7f18b0157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.768+0000 7f18b5800640 1 -- 172.21.15.47:0/1572226902 >> 172.21.15.47:0/1572226902 conn(0x7f18b00cfd00 msgr2=0x7f18b015bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.768+0000 7f18b5800640 1 -- 172.21.15.47:0/1572226902 shutdown_connections 2024-04-24T22:44:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:12.768+0000 7f18b5800640 1 -- 172.21.15.47:0/1572226902 wait complete. 2024-04-24T22:44:12.943 INFO:tasks.ceph:Scrubbing osd.3 2024-04-24T22:44:12.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-24T22:44:13.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.167+0000 7f8d47800640 1 Processor -- start 2024-04-24T22:44:13.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.167+0000 7f8d47800640 1 -- start start 2024-04-24T22:44:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d47800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d40152600 0x7f8d4015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d47800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d400c9270 0x7f8d40158d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d47800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d400c6650 0x7f8d40072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d47800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8d401592a0 con 0x7f8d40152600 2024-04-24T22:44:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d47800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8d400c6010 con 0x7f8d400c6650 2024-04-24T22:44:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d47800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8d400c57e0 con 0x7f8d400c9270 2024-04-24T22:44:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d44e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d40152600 0x7f8d4015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d44e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d40152600 0x7f8d4015c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35730/0 (socket says 172.21.15.47:35730) 2024-04-24T22:44:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d44e00640 1 -- 172.21.15.47:0/1771382014 learned_addr learned my addr 172.21.15.47:0/1771382014 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d45800640 1 --2- 172.21.15.47:0/1771382014 >> v2:172.21.15.113:3300/0 conn(0x7f8d400c6650 0x7f8d40072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/1771382014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d400c9270 0x7f8d40158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d44e00640 1 -- 172.21.15.47:0/1771382014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d400c9270 msgr2=0x7f8d40158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d44e00640 1 --2- 172.21.15.47:0/1771382014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d400c9270 0x7f8d40158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d44e00640 1 -- 172.21.15.47:0/1771382014 >> v2:172.21.15.113:3300/0 conn(0x7f8d400c6650 msgr2=0x7f8d40072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d44e00640 1 --2- 172.21.15.47:0/1771382014 >> v2:172.21.15.113:3300/0 conn(0x7f8d400c6650 0x7f8d40072bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d44e00640 1 -- 172.21.15.47:0/1771382014 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d401529e0 con 0x7f8d40152600 2024-04-24T22:44:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.170+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/1771382014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d400c9270 0x7f8d40158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:44:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.171+0000 7f8d44e00640 1 --2- 172.21.15.47:0/1771382014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40152600 0x7f8d4015c4e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d40070970 tx=0x7f8d3002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b7150134aef3a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.171+0000 7f8d3f400640 1 -- 172.21.15.47:0/1771382014 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30036070 con 0x7f8d40152600 2024-04-24T22:44:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.171+0000 7f8d3f400640 1 -- 172.21.15.47:0/1771382014 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d30031600 con 0x7f8d40152600 2024-04-24T22:44:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.171+0000 7f8d3f400640 1 -- 172.21.15.47:0/1771382014 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d300319d0 con 0x7f8d40152600 2024-04-24T22:44:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.172+0000 7f8d47800640 1 -- 172.21.15.47:0/1771382014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40152600 msgr2=0x7f8d4015c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.172+0000 7f8d47800640 1 --2- 172.21.15.47:0/1771382014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40152600 0x7f8d4015c4e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d40070970 tx=0x7f8d3002a660 comp rx=0 tx=0).stop 2024-04-24T22:44:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.172+0000 7f8d47800640 1 -- 172.21.15.47:0/1771382014 shutdown_connections 2024-04-24T22:44:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.172+0000 7f8d47800640 1 --2- 172.21.15.47:0/1771382014 >> v2:172.21.15.113:3300/0 conn(0x7f8d400c6650 0x7f8d40072bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.172+0000 7f8d47800640 1 --2- 172.21.15.47:0/1771382014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d400c9270 0x7f8d40158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.172+0000 7f8d47800640 1 --2- 172.21.15.47:0/1771382014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40152600 0x7f8d4015c4e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.172+0000 7f8d47800640 1 -- 172.21.15.47:0/1771382014 >> 172.21.15.47:0/1771382014 conn(0x7f8d400c86a0 msgr2=0x7f8d400c8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.172+0000 7f8d47800640 1 -- 172.21.15.47:0/1771382014 shutdown_connections 2024-04-24T22:44:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.172+0000 7f8d47800640 1 -- 172.21.15.47:0/1771382014 wait complete. 2024-04-24T22:44:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.174+0000 7f8d47800640 1 Processor -- start 2024-04-24T22:44:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.174+0000 7f8d47800640 1 -- start start 2024-04-24T22:44:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d47800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d400c6650 0x7f8d401e1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d47800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d400c9270 0x7f8d40152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d47800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d40152600 0x7f8d401e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d47800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4015ba00 con 0x7f8d400c9270 2024-04-24T22:44:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d47800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8d400c5d30 con 0x7f8d400c6650 2024-04-24T22:44:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d47800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4015bce0 con 0x7f8d40152600 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d400c9270 0x7f8d40152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d44e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d400c6650 0x7f8d401e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d44e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d400c6650 0x7f8d401e1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60324/0 (socket says 172.21.15.47:60324) 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d45800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d40152600 0x7f8d401e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d44e00640 1 -- 172.21.15.47:0/230421386 learned_addr learned my addr 172.21.15.47:0/230421386 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d44e00640 1 -- 172.21.15.47:0/230421386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d40152600 msgr2=0x7f8d401e22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d44e00640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d40152600 0x7f8d401e22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d44e00640 1 -- 172.21.15.47:0/230421386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d400c9270 msgr2=0x7f8d40152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d44e00640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d400c9270 0x7f8d40152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.175+0000 7f8d44e00640 1 -- 172.21.15.47:0/230421386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d401e12e0 con 0x7f8d400c6650 2024-04-24T22:44:13.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.176+0000 7f8d44e00640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d400c6650 0x7f8d401e1750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d30000f80 tx=0x7f8d30004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.176+0000 7f8d3cc00640 1 -- 172.21.15.47:0/230421386 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30036050 con 0x7f8d400c6650 2024-04-24T22:44:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.176+0000 7f8d3cc00640 1 -- 172.21.15.47:0/230421386 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d30042070 con 0x7f8d400c6650 2024-04-24T22:44:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.176+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d401ee860 con 0x7f8d400c6650 2024-04-24T22:44:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.176+0000 7f8d3cc00640 1 -- 172.21.15.47:0/230421386 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30009510 con 0x7f8d400c6650 2024-04-24T22:44:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.176+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d401e2830 con 0x7f8d400c6650 2024-04-24T22:44:13.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.177+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d401ee650 con 0x7f8d400c6650 2024-04-24T22:44:13.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.178+0000 7f8d3cc00640 1 -- 172.21.15.47:0/230421386 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d30007c70 con 0x7f8d400c6650 2024-04-24T22:44:13.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.179+0000 7f8d3cc00640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8d2006be60 0x7f8d2006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.180+0000 7f8d3cc00640 1 -- 172.21.15.47:0/230421386 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8d300abbb0 con 0x7f8d400c6650 2024-04-24T22:44:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.180+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8d2006be60 0x7f8d2006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.180+0000 7f8d3cc00640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] conn(0x7f8d20074180 0x7f8d20076600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.180+0000 7f8d3cc00640 1 -- 172.21.15.47:0/230421386 --> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d3004ae80 con 0x7f8d20074180 2024-04-24T22:44:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.180+0000 7f8d3cc00640 1 -- 172.21.15.47:0/230421386 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8d300b02a0 con 0x7f8d400c6650 2024-04-24T22:44:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.180+0000 7f8d45800640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] conn(0x7f8d20074180 0x7f8d20076600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.180+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8d2006be60 0x7f8d2006e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d40151f00 tx=0x7f8d2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.181+0000 7f8d45800640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] conn(0x7f8d20074180 0x7f8d20076600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.182+0000 7f8d3cc00640 1 -- 172.21.15.47:0/230421386 <== osd.3 v2:172.21.15.47:6824/872127764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d3004ae80 con 0x7f8d20074180 2024-04-24T22:44:13.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.196+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 --> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8d40073c00 con 0x7f8d20074180 2024-04-24T22:44:13.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.197+0000 7f8d3cc00640 1 -- 172.21.15.47:0/230421386 <== osd.3 v2:172.21.15.47:6824/872127764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8d40073c00 con 0x7f8d20074180 2024-04-24T22:44:13.200 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:44:13.200 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T22:44:13.200 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:44:13.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.197+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 >> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] conn(0x7f8d20074180 msgr2=0x7f8d20076600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.197+0000 7f8d47800640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.47:6824/872127764,v1:172.21.15.47:6825/872127764] conn(0x7f8d20074180 0x7f8d20076600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.198+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8d2006be60 msgr2=0x7f8d2006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.198+0000 7f8d47800640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8d2006be60 0x7f8d2006e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d40151f00 tx=0x7f8d2c008040 comp rx=0 tx=0).stop 2024-04-24T22:44:13.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.198+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d400c6650 msgr2=0x7f8d401e1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.198+0000 7f8d47800640 1 --2- 172.21.15.47:0/230421386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d400c6650 0x7f8d401e1750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d30000f80 tx=0x7f8d30004a40 comp rx=0 tx=0).stop 2024-04-24T22:44:13.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.198+0000 7f8d45800640 1 -- 172.21.15.47:0/230421386 reap_dead start 2024-04-24T22:44:13.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.198+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 shutdown_connections 2024-04-24T22:44:13.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.198+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 >> 172.21.15.47:0/230421386 conn(0x7f8d400c86a0 msgr2=0x7f8d4013e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:13.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.198+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 shutdown_connections 2024-04-24T22:44:13.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.198+0000 7f8d47800640 1 -- 172.21.15.47:0/230421386 wait complete. 2024-04-24T22:44:13.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-24T22:44:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.435+0000 7f3aeac00640 1 Processor -- start 2024-04-24T22:44:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.435+0000 7f3aeac00640 1 -- start start 2024-04-24T22:44:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.436+0000 7f3aeac00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ae40c1a00 0x7f3ae4072f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.436+0000 7f3aeac00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae40738f0 con 0x7f3ae4151780 2024-04-24T22:44:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.436+0000 7f3aeac00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae415aa30 con 0x7f3ae40734a0 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.436+0000 7f3aeac00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae415ad10 con 0x7f3ae40c1a00 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.437+0000 7f3ae3e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3ae4151780 0x7f3ae40c8590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46674/0 (socket says 172.21.15.47:46674) 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.437+0000 7f3ae3e00640 1 -- 172.21.15.47:0/3660965459 learned_addr learned my addr 172.21.15.47:0/3660965459 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.437+0000 7f3ae3400640 1 --2- 172.21.15.47:0/3660965459 >> v2:172.21.15.113:3300/0 conn(0x7f3ae40c1a00 0x7f3ae4072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.437+0000 7f3ae2a00640 1 -- 172.21.15.47:0/3660965459 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2502290354 0 0) 0x7f3acc001110 con 0x7f3ae4151780 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.437+0000 7f3ae2a00640 1 -- 172.21.15.47:0/3660965459 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ac8005730 con 0x7f3ae4151780 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.437+0000 7f3ae2a00640 1 -- 172.21.15.47:0/3660965459 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087438987 0 0) 0x7f3ae415aa30 con 0x7f3ae40734a0 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.437+0000 7f3ae2a00640 1 -- 172.21.15.47:0/3660965459 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3acc001110 con 0x7f3ae40734a0 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.437+0000 7f3ae3400640 1 -- 172.21.15.47:0/3660965459 >> v1:172.21.15.113:6789/0 conn(0x7f3ae40734a0 legacy=0x7f3ae41576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.437+0000 7f3ae3400640 1 -- 172.21.15.47:0/3660965459 >> v1:172.21.15.47:6789/0 conn(0x7f3ae4151780 legacy=0x7f3ae40c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.437+0000 7f3ae3400640 1 -- 172.21.15.47:0/3660965459 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae415aff0 con 0x7f3ae40c1a00 2024-04-24T22:44:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.438+0000 7f3ae3400640 1 --2- 172.21.15.47:0/3660965459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40c1a00 0x7f3ae4072f60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009d90 tx=0x7f3ad402a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=214297e6b0fbc34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.439+0000 7f3ae2a00640 1 -- 172.21.15.47:0/3660965459 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4037070 con 0x7f3ae40c1a00 2024-04-24T22:44:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.439+0000 7f3ae2a00640 1 -- 172.21.15.47:0/3660965459 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad4007740 con 0x7f3ae40c1a00 2024-04-24T22:44:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.439+0000 7f3ae2a00640 1 -- 172.21.15.47:0/3660965459 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007b10 con 0x7f3ae40c1a00 2024-04-24T22:44:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.439+0000 7f3aeac00640 1 -- 172.21.15.47:0/3660965459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40c1a00 msgr2=0x7f3ae4072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.439+0000 7f3aeac00640 1 --2- 172.21.15.47:0/3660965459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40c1a00 0x7f3ae4072f60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009d90 tx=0x7f3ad402a780 comp rx=0 tx=0).stop 2024-04-24T22:44:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.439+0000 7f3aeac00640 1 -- 172.21.15.47:0/3660965459 shutdown_connections 2024-04-24T22:44:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.439+0000 7f3aeac00640 1 --2- 172.21.15.47:0/3660965459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40c1a00 0x7f3ae4072f60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.439+0000 7f3aeac00640 1 -- 172.21.15.47:0/3660965459 >> 172.21.15.47:0/3660965459 conn(0x7f3ae406e580 msgr2=0x7f3ae406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.439+0000 7f3aeac00640 1 -- 172.21.15.47:0/3660965459 shutdown_connections 2024-04-24T22:44:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.439+0000 7f3aeac00640 1 -- 172.21.15.47:0/3660965459 wait complete. 2024-04-24T22:44:13.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.441+0000 7f3aeac00640 1 Processor -- start 2024-04-24T22:44:13.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.441+0000 7f3aeac00640 1 -- start start 2024-04-24T22:44:13.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.442+0000 7f3aeac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40734a0 0x7f3ae40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.442+0000 7f3aeac00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ae40c1a00 0x7f3ae40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.442+0000 7f3aeac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae4151780 0x7f3ae4147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.442+0000 7f3aeac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae40c8ca0 con 0x7f3ae4151780 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.442+0000 7f3aeac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae415b5b0 con 0x7f3ae40734a0 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.442+0000 7f3aeac00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae40c8e20 con 0x7f3ae40c1a00 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40734a0 0x7f3ae40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ae40c1a00 0x7f3ae40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ae40c1a00 0x7f3ae40c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:34956/0 (socket says 172.21.15.47:34956) 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40734a0 0x7f3ae40ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:60352/0 (socket says 172.21.15.47:60352) 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3400640 1 -- 172.21.15.47:0/1415086019 learned_addr learned my addr 172.21.15.47:0/1415086019 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae8c00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae4151780 0x7f3ae4147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3400640 1 -- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40734a0 msgr2=0x7f3ae40ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3400640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40734a0 0x7f3ae40ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3400640 1 -- 172.21.15.47:0/1415086019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae4151780 msgr2=0x7f3ae4147920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3400640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae4151780 0x7f3ae4147920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3400640 1 -- 172.21.15.47:0/1415086019 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae40c5220 con 0x7f3ae40c1a00 2024-04-24T22:44:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae3e00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40734a0 0x7f3ae40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.443+0000 7f3ae8c00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae4151780 0x7f3ae4147920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.444+0000 7f3ae3400640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ae40c1a00 0x7f3ae40c99b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009d90 tx=0x7f3ad4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.444+0000 7f3adbe00640 1 -- 172.21.15.47:0/1415086019 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4042020 con 0x7f3ae40c1a00 2024-04-24T22:44:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.444+0000 7f3adbe00640 1 -- 172.21.15.47:0/1415086019 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad4032870 con 0x7f3ae40c1a00 2024-04-24T22:44:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.444+0000 7f3adbe00640 1 -- 172.21.15.47:0/1415086019 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4037040 con 0x7f3ae40c1a00 2024-04-24T22:44:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.444+0000 7f3aeac00640 1 -- 172.21.15.47:0/1415086019 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae40c5500 con 0x7f3ae40c1a00 2024-04-24T22:44:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.444+0000 7f3aeac00640 1 -- 172.21.15.47:0/1415086019 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae40c5ac0 con 0x7f3ae40c1a00 2024-04-24T22:44:13.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.445+0000 7f3aeac00640 1 -- 172.21.15.47:0/1415086019 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae4073b90 con 0x7f3ae40c1a00 2024-04-24T22:44:13.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.446+0000 7f3adbe00640 1 -- 172.21.15.47:0/1415086019 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ad4045020 con 0x7f3ae40c1a00 2024-04-24T22:44:13.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.448+0000 7f3adbe00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ac006be60 0x7f3ac006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.448+0000 7f3adbe00640 1 -- 172.21.15.47:0/1415086019 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f3ad40abc50 con 0x7f3ae40c1a00 2024-04-24T22:44:13.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.448+0000 7f3ae3e00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ac006be60 0x7f3ac006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.448+0000 7f3ae3e00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ac006be60 0x7f3ac006e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3acc000f60 tx=0x7f3acc007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.453+0000 7f3adbe00640 1 -- 172.21.15.47:0/1415086019 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ad404c070 con 0x7f3ae40c1a00 2024-04-24T22:44:13.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.661+0000 7f3aeac00640 1 -- 172.21.15.47:0/1415086019 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f3ae40c4fd0 con 0x7f3ac006be60 2024-04-24T22:44:13.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.662+0000 7f3adbe00640 1 -- 172.21.15.47:0/1415086019 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3ae40c4fd0 con 0x7f3ac006be60 2024-04-24T22:44:13.663 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 3 to deep-scrub 2024-04-24T22:44:13.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 -- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ac006be60 msgr2=0x7f3ac006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ac006be60 0x7f3ac006e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3acc000f60 tx=0x7f3acc007d70 comp rx=0 tx=0).stop 2024-04-24T22:44:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 -- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ae40c1a00 msgr2=0x7f3ae40c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ae40c1a00 0x7f3ae40c99b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009d90 tx=0x7f3ad4004a40 comp rx=0 tx=0).stop 2024-04-24T22:44:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 -- 172.21.15.47:0/1415086019 shutdown_connections 2024-04-24T22:44:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f3ac006be60 0x7f3ac006e320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae4151780 0x7f3ae4147920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ae40c1a00 0x7f3ae40c99b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 --2- 172.21.15.47:0/1415086019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ae40734a0 0x7f3ae40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 -- 172.21.15.47:0/1415086019 >> 172.21.15.47:0/1415086019 conn(0x7f3ae406e580 msgr2=0x7f3ae406b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:13.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 -- 172.21.15.47:0/1415086019 shutdown_connections 2024-04-24T22:44:13.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.665+0000 7f3aeac00640 1 -- 172.21.15.47:0/1415086019 wait complete. 2024-04-24T22:44:13.687 INFO:tasks.ceph:Scrubbing osd.4 2024-04-24T22:44:13.687 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-24T22:44:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.909+0000 7fd0fee00640 1 Processor -- start 2024-04-24T22:44:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.909+0000 7fd0fee00640 1 -- start start 2024-04-24T22:44:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.911+0000 7fd0fee00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd0f8150cc0 0x7fd0f80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.911+0000 7fd0fee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0f8071870 0x7fd0f80c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.911+0000 7fd0fee00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f8156a80 con 0x7fd0f80c7b30 2024-04-24T22:44:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.911+0000 7fd0fee00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f815a000 con 0x7fd0f8150cc0 2024-04-24T22:44:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.911+0000 7fd0fee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f815a2d0 con 0x7fd0f8071870 2024-04-24T22:44:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.911+0000 7fd0f7400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0f8071870 0x7fd0f80c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.911+0000 7fd0f7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd0f8150cc0 0x7fd0f80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.911+0000 7fd0fce00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd0f80c7b30 0x7fd0f8156f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46686/0 (socket says 172.21.15.47:46686) 2024-04-24T22:44:13.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.911+0000 7fd0fce00640 1 -- 172.21.15.47:0/1019946972 learned_addr learned my addr 172.21.15.47:0/1019946972 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:13.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.912+0000 7fd0f7e00640 1 -- 172.21.15.47:0/1019946972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0f8071870 msgr2=0x7fd0f80c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:13.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.912+0000 7fd0f7e00640 1 --2- 172.21.15.47:0/1019946972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0f8071870 0x7fd0f80c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.912+0000 7fd0f7e00640 1 -- 172.21.15.47:0/1019946972 >> v1:172.21.15.47:6789/0 conn(0x7fd0f80c7b30 legacy=0x7fd0f8156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.912+0000 7fd0f7e00640 1 -- 172.21.15.47:0/1019946972 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f815a5b0 con 0x7fd0f8150cc0 2024-04-24T22:44:13.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.913+0000 7fd0f7e00640 1 --2- 172.21.15.47:0/1019946972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0f8150cc0 0x7fd0f80709e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009e10 tx=0x7fd0e802a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9368c67db9de3d2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.913+0000 7fd0f6a00640 1 -- 172.21.15.47:0/1019946972 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8007740 con 0x7fd0f8150cc0 2024-04-24T22:44:13.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.913+0000 7fd0f6a00640 1 -- 172.21.15.47:0/1019946972 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e80078e0 con 0x7fd0f8150cc0 2024-04-24T22:44:13.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.913+0000 7fd0f6a00640 1 -- 172.21.15.47:0/1019946972 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8007cb0 con 0x7fd0f8150cc0 2024-04-24T22:44:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.914+0000 7fd0fee00640 1 -- 172.21.15.47:0/1019946972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0f8150cc0 msgr2=0x7fd0f80709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.914+0000 7fd0fee00640 1 --2- 172.21.15.47:0/1019946972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0f8150cc0 0x7fd0f80709e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009e10 tx=0x7fd0e802a6d0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.914+0000 7fd0fee00640 1 -- 172.21.15.47:0/1019946972 shutdown_connections 2024-04-24T22:44:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.914+0000 7fd0fee00640 1 --2- 172.21.15.47:0/1019946972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0f8071870 0x7fd0f80c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.914+0000 7fd0fee00640 1 --2- 172.21.15.47:0/1019946972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0f8150cc0 0x7fd0f80709e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.914+0000 7fd0fee00640 1 -- 172.21.15.47:0/1019946972 >> 172.21.15.47:0/1019946972 conn(0x7fd0f80c86a0 msgr2=0x7fd0f80c8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.914+0000 7fd0fee00640 1 -- 172.21.15.47:0/1019946972 shutdown_connections 2024-04-24T22:44:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.914+0000 7fd0fee00640 1 -- 172.21.15.47:0/1019946972 wait complete. 2024-04-24T22:44:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.916+0000 7fd0fee00640 1 Processor -- start 2024-04-24T22:44:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.916+0000 7fd0fee00640 1 -- start start 2024-04-24T22:44:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0fee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0f8071870 0x7fd0f81f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0fee00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0f80c7b30 0x7fd0f81e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0fee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0f8150cc0 0x7fd0f814bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0fee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f815ae50 con 0x7fd0f8150cc0 2024-04-24T22:44:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0fee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f815ab70 con 0x7fd0f8071870 2024-04-24T22:44:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0fee00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f81510a0 con 0x7fd0f80c7b30 2024-04-24T22:44:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0fce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0f8150cc0 0x7fd0f814bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0fce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0f8150cc0 0x7fd0f814bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35762/0 (socket says 172.21.15.47:35762) 2024-04-24T22:44:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0fce00640 1 -- 172.21.15.47:0/2336999071 learned_addr learned my addr 172.21.15.47:0/2336999071 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0f7e00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0f8071870 0x7fd0f81f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.917+0000 7fd0f7400640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0f80c7b30 0x7fd0f81e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0fce00640 1 -- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0f80c7b30 msgr2=0x7fd0f81e0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0fce00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0f80c7b30 0x7fd0f81e0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0fce00640 1 -- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0f8071870 msgr2=0x7fd0f81f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0fce00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0f8071870 0x7fd0f81f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0fce00640 1 -- 172.21.15.47:0/2336999071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f81f27f0 con 0x7fd0f8150cc0 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0f7e00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0f8071870 0x7fd0f81f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0fce00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0f8150cc0 0x7fd0f814bd00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000c3c0 tx=0x7fd0f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0efe00640 1 -- 172.21.15.47:0/2336999071 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0017070 con 0x7fd0f8150cc0 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0efe00640 1 -- 172.21.15.47:0/2336999071 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f0005ce0 con 0x7fd0f8150cc0 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f8154a00 con 0x7fd0f8150cc0 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0efe00640 1 -- 172.21.15.47:0/2336999071 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00072f0 con 0x7fd0f8150cc0 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.918+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f8154fc0 con 0x7fd0f8150cc0 2024-04-24T22:44:13.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.919+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd0f8073c00 con 0x7fd0f8150cc0 2024-04-24T22:44:13.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.921+0000 7fd0efe00640 1 -- 172.21.15.47:0/2336999071 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0f001a070 con 0x7fd0f8150cc0 2024-04-24T22:44:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.921+0000 7fd0efe00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd0d406be30 0x7fd0d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.922+0000 7fd0efe00640 1 -- 172.21.15.47:0/2336999071 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd0f008a290 con 0x7fd0f8150cc0 2024-04-24T22:44:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.922+0000 7fd0efe00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] conn(0x7fd0d4074150 0x7fd0d40765d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.922+0000 7fd0efe00640 1 -- 172.21.15.47:0/2336999071 --> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0f0014ec0 con 0x7fd0d4074150 2024-04-24T22:44:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.922+0000 7fd0efe00640 1 -- 172.21.15.47:0/2336999071 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd0f0091050 con 0x7fd0f8150cc0 2024-04-24T22:44:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.922+0000 7fd0f7e00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd0d406be30 0x7fd0d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.922+0000 7fd0f7400640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] conn(0x7fd0d4074150 0x7fd0d40765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:13.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.923+0000 7fd0f7e00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd0d406be30 0x7fd0d406e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8034000 tx=0x7fd0e802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.923+0000 7fd0f7400640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] conn(0x7fd0d4074150 0x7fd0d40765d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:13.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.924+0000 7fd0efe00640 1 -- 172.21.15.47:0/2336999071 <== osd.4 v2:172.21.15.113:6817/2737975939 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0f0014ec0 con 0x7fd0d4074150 2024-04-24T22:44:13.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.939+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 --> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd0f8154620 con 0x7fd0d4074150 2024-04-24T22:44:13.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.939+0000 7fd0efe00640 1 -- 172.21.15.47:0/2336999071 <== osd.4 v2:172.21.15.113:6817/2737975939 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd0f8154620 con 0x7fd0d4074150 2024-04-24T22:44:13.941 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:44:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T22:44:13.941 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:44:13.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] conn(0x7fd0d4074150 msgr2=0x7fd0d40765d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fee00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:6817/2737975939,v1:172.21.15.113:6819/2737975939] conn(0x7fd0d4074150 0x7fd0d40765d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:13.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd0d406be30 msgr2=0x7fd0d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fee00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fd0d406be30 0x7fd0d406e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8034000 tx=0x7fd0e802f000 comp rx=0 tx=0).stop 2024-04-24T22:44:13.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0f8150cc0 msgr2=0x7fd0f814bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fee00640 1 --2- 172.21.15.47:0/2336999071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0f8150cc0 0x7fd0f814bd00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000c3c0 tx=0x7fd0f000c880 comp rx=0 tx=0).stop 2024-04-24T22:44:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fce00640 1 -- 172.21.15.47:0/2336999071 reap_dead start 2024-04-24T22:44:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 shutdown_connections 2024-04-24T22:44:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 >> 172.21.15.47:0/2336999071 conn(0x7fd0f80c86a0 msgr2=0x7fd0f814d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 shutdown_connections 2024-04-24T22:44:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:13.940+0000 7fd0fee00640 1 -- 172.21.15.47:0/2336999071 wait complete. 2024-04-24T22:44:13.960 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-24T22:44:14.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.171+0000 7f3142e00640 1 Processor -- start 2024-04-24T22:44:14.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.171+0000 7f3142e00640 1 -- start start 2024-04-24T22:44:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.173+0000 7f3142e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 0x7f313c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.173+0000 7f3142e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313c15c3d0 con 0x7f313c072c60 2024-04-24T22:44:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.173+0000 7f3142e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313c15c6b0 con 0x7f313c073040 2024-04-24T22:44:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.173+0000 7f3142e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f313c158bf0 con 0x7f313c06bb90 2024-04-24T22:44:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.173+0000 7f3141a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f313c072c60 0x7f313c0c2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46696/0 (socket says 172.21.15.47:46696) 2024-04-24T22:44:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.173+0000 7f3141a00640 1 -- 172.21.15.47:0/1895197966 learned_addr learned my addr 172.21.15.47:0/1895197966 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.173+0000 7f3142400640 1 --2- 172.21.15.47:0/1895197966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 0x7f313c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.174+0000 7f313be00640 1 -- 172.21.15.47:0/1895197966 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1563655730 0 0) 0x7f313c15c6b0 con 0x7f313c073040 2024-04-24T22:44:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.174+0000 7f313be00640 1 -- 172.21.15.47:0/1895197966 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3124003520 con 0x7f313c073040 2024-04-24T22:44:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.174+0000 7f3142400640 1 -- 172.21.15.47:0/1895197966 >> v1:172.21.15.113:6789/0 conn(0x7f313c073040 legacy=0x7f313c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.174+0000 7f3142400640 1 -- 172.21.15.47:0/1895197966 >> v1:172.21.15.47:6789/0 conn(0x7f313c072c60 legacy=0x7f313c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.174+0000 7f313be00640 1 -- 172.21.15.47:0/1895197966 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3641210774 0 0) 0x7f313c15c3d0 con 0x7f313c072c60 2024-04-24T22:44:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.174+0000 7f3142400640 1 -- 172.21.15.47:0/1895197966 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c1585a0 con 0x7f313c06bb90 2024-04-24T22:44:14.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.175+0000 7f3142400640 1 --2- 172.21.15.47:0/1895197966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 0x7f313c15b860 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3134009dc0 tx=0x7f313402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa03490c735855b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:14.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.175+0000 7f313be00640 1 -- 172.21.15.47:0/1895197966 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134007740 con 0x7f313c06bb90 2024-04-24T22:44:14.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.175+0000 7f313be00640 1 -- 172.21.15.47:0/1895197966 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31340078e0 con 0x7f313c06bb90 2024-04-24T22:44:14.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.175+0000 7f313be00640 1 -- 172.21.15.47:0/1895197966 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134007cb0 con 0x7f313c06bb90 2024-04-24T22:44:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.176+0000 7f3142e00640 1 -- 172.21.15.47:0/1895197966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 msgr2=0x7f313c15b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.176+0000 7f3142e00640 1 --2- 172.21.15.47:0/1895197966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 0x7f313c15b860 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3134009dc0 tx=0x7f313402a6d0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.176+0000 7f3142e00640 1 -- 172.21.15.47:0/1895197966 shutdown_connections 2024-04-24T22:44:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.176+0000 7f3142e00640 1 --2- 172.21.15.47:0/1895197966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 0x7f313c15b860 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.176+0000 7f3142e00640 1 -- 172.21.15.47:0/1895197966 >> 172.21.15.47:0/1895197966 conn(0x7f313c0cb900 msgr2=0x7f313c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.176+0000 7f3142e00640 1 -- 172.21.15.47:0/1895197966 shutdown_connections 2024-04-24T22:44:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.176+0000 7f3142e00640 1 -- 172.21.15.47:0/1895197966 wait complete. 2024-04-24T22:44:14.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.178+0000 7f3142e00640 1 Processor -- start 2024-04-24T22:44:14.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.178+0000 7f3142e00640 1 -- start start 2024-04-24T22:44:14.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3142e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 0x7f313c1f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3142e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c072c60 0x7f313c13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3142e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f313c073040 0x7f313c13dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3142e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f313c159080 con 0x7f313c073040 2024-04-24T22:44:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3142e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f313c158270 con 0x7f313c072c60 2024-04-24T22:44:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3142e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f313c159200 con 0x7f313c06bb90 2024-04-24T22:44:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3142400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f313c073040 0x7f313c13dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3142400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f313c073040 0x7f313c13dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35768/0 (socket says 172.21.15.47:35768) 2024-04-24T22:44:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3141000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c072c60 0x7f313c13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3141a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 0x7f313c1f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.179+0000 7f3142400640 1 -- 172.21.15.47:0/979221764 learned_addr learned my addr 172.21.15.47:0/979221764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3141000640 1 -- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 msgr2=0x7f313c1f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3141000640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 0x7f313c1f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3141000640 1 -- 172.21.15.47:0/979221764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f313c073040 msgr2=0x7f313c13dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3141000640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f313c073040 0x7f313c13dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3141000640 1 -- 172.21.15.47:0/979221764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c13eb60 con 0x7f313c072c60 2024-04-24T22:44:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3142400640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f313c073040 0x7f313c13dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3141000640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c072c60 0x7f313c13fd30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f312800c430 tx=0x7f312800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3139600640 1 -- 172.21.15.47:0/979221764 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128017070 con 0x7f313c072c60 2024-04-24T22:44:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3139600640 1 -- 172.21.15.47:0/979221764 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31280052d0 con 0x7f313c072c60 2024-04-24T22:44:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3139600640 1 -- 172.21.15.47:0/979221764 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128003830 con 0x7f313c072c60 2024-04-24T22:44:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3142e00640 1 -- 172.21.15.47:0/979221764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c0c9090 con 0x7f313c072c60 2024-04-24T22:44:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.180+0000 7f3142e00640 1 -- 172.21.15.47:0/979221764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c072070 con 0x7f313c072c60 2024-04-24T22:44:14.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.182+0000 7f3142e00640 1 -- 172.21.15.47:0/979221764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313c1eff20 con 0x7f313c072c60 2024-04-24T22:44:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.183+0000 7f3139600640 1 -- 172.21.15.47:0/979221764 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f312801a070 con 0x7f313c072c60 2024-04-24T22:44:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.183+0000 7f3139600640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f311c06be30 0x7f311c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.184+0000 7f3139600640 1 -- 172.21.15.47:0/979221764 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f312808a2b0 con 0x7f313c072c60 2024-04-24T22:44:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.184+0000 7f3141a00640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f311c06be30 0x7f311c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.184+0000 7f3141a00640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f311c06be30 0x7f311c06e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3130007f70 tx=0x7f3130007f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:14.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.188+0000 7f3139600640 1 -- 172.21.15.47:0/979221764 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31280584f0 con 0x7f313c072c60 2024-04-24T22:44:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.391+0000 7f3142e00640 1 -- 172.21.15.47:0/979221764 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f313c068d50 con 0x7f311c06be30 2024-04-24T22:44:14.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.393+0000 7f3139600640 1 -- 172.21.15.47:0/979221764 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f313c068d50 con 0x7f311c06be30 2024-04-24T22:44:14.394 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 4 to deep-scrub 2024-04-24T22:44:14.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.395+0000 7f3142e00640 1 -- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f311c06be30 msgr2=0x7f311c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.395+0000 7f3142e00640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f311c06be30 0x7f311c06e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3130007f70 tx=0x7f3130007f00 comp rx=0 tx=0).stop 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.396+0000 7f3142e00640 1 -- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c072c60 msgr2=0x7f313c13fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.396+0000 7f3142e00640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c072c60 0x7f313c13fd30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f312800c430 tx=0x7f312800c8f0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.396+0000 7f3142e00640 1 -- 172.21.15.47:0/979221764 shutdown_connections 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.396+0000 7f3142e00640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f311c06be30 0x7f311c06e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.396+0000 7f3142e00640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f313c073040 0x7f313c13dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.396+0000 7f3142e00640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f313c072c60 0x7f313c13fd30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.396+0000 7f3142e00640 1 --2- 172.21.15.47:0/979221764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f313c06bb90 0x7f313c1f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.396+0000 7f3142e00640 1 -- 172.21.15.47:0/979221764 >> 172.21.15.47:0/979221764 conn(0x7f313c0cb900 msgr2=0x7f313c1556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.396+0000 7f3142e00640 1 -- 172.21.15.47:0/979221764 shutdown_connections 2024-04-24T22:44:14.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.396+0000 7f3142e00640 1 -- 172.21.15.47:0/979221764 wait complete. 2024-04-24T22:44:14.417 INFO:tasks.ceph:Scrubbing osd.5 2024-04-24T22:44:14.417 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-24T22:44:14.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.625+0000 7fb4df000640 1 Processor -- start 2024-04-24T22:44:14.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.625+0000 7fb4df000640 1 -- start start 2024-04-24T22:44:14.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.628+0000 7fb4df000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb4d8150b70 0x7fb4d80c7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.628+0000 7fb4df000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4d8150f50 0x7fb4d80cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.628+0000 7fb4df000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d815a450 con 0x7fb4d80625a0 2024-04-24T22:44:14.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.628+0000 7fb4df000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d80c8a10 con 0x7fb4d8150b70 2024-04-24T22:44:14.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.628+0000 7fb4df000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d815a730 con 0x7fb4d8150f50 2024-04-24T22:44:14.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.628+0000 7fb4de600640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb4d80625a0 0x7fb4d815d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46710/0 (socket says 172.21.15.47:46710) 2024-04-24T22:44:14.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.628+0000 7fb4ddc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb4d8150b70 0x7fb4d80c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.628+0000 7fb4de600640 1 -- 172.21.15.47:0/265442625 learned_addr learned my addr 172.21.15.47:0/265442625 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:14.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.628+0000 7fb4dd200640 1 --2- 172.21.15.47:0/265442625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4d8150f50 0x7fb4d80cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.629+0000 7fb4ddc00640 1 -- 172.21.15.47:0/265442625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4d8150f50 msgr2=0x7fb4d80cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:14.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.629+0000 7fb4ddc00640 1 --2- 172.21.15.47:0/265442625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4d8150f50 0x7fb4d80cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.629+0000 7fb4ddc00640 1 -- 172.21.15.47:0/265442625 >> v1:172.21.15.47:6789/0 conn(0x7fb4d80625a0 legacy=0x7fb4d815d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.629+0000 7fb4ddc00640 1 -- 172.21.15.47:0/265442625 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d815aa10 con 0x7fb4d8150b70 2024-04-24T22:44:14.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.629+0000 7fb4d7e00640 1 -- 172.21.15.47:0/265442625 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860014262 0 0) 0x7fb4d815a450 con 0x7fb4d80625a0 2024-04-24T22:44:14.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.630+0000 7fb4ddc00640 1 --2- 172.21.15.47:0/265442625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4d8150b70 0x7fb4d80c7460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097a0 tx=0x7fb4cc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=513fa4d80dc18860 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:14.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.630+0000 7fb4d7e00640 1 -- 172.21.15.47:0/265442625 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc037070 con 0x7fb4d8150b70 2024-04-24T22:44:14.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.630+0000 7fb4d7e00640 1 -- 172.21.15.47:0/265442625 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc02ae50 con 0x7fb4d8150b70 2024-04-24T22:44:14.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.630+0000 7fb4d7e00640 1 -- 172.21.15.47:0/265442625 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc032660 con 0x7fb4d8150b70 2024-04-24T22:44:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.631+0000 7fb4df000640 1 -- 172.21.15.47:0/265442625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4d8150b70 msgr2=0x7fb4d80c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.631+0000 7fb4df000640 1 --2- 172.21.15.47:0/265442625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4d8150b70 0x7fb4d80c7460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0097a0 tx=0x7fb4cc02a100 comp rx=0 tx=0).stop 2024-04-24T22:44:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.631+0000 7fb4df000640 1 -- 172.21.15.47:0/265442625 shutdown_connections 2024-04-24T22:44:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.631+0000 7fb4df000640 1 --2- 172.21.15.47:0/265442625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4d8150f50 0x7fb4d80cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.631+0000 7fb4df000640 1 --2- 172.21.15.47:0/265442625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4d8150b70 0x7fb4d80c7460 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.631+0000 7fb4df000640 1 -- 172.21.15.47:0/265442625 >> 172.21.15.47:0/265442625 conn(0x7fb4d806b160 msgr2=0x7fb4d806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.631+0000 7fb4df000640 1 -- 172.21.15.47:0/265442625 shutdown_connections 2024-04-24T22:44:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.631+0000 7fb4df000640 1 -- 172.21.15.47:0/265442625 wait complete. 2024-04-24T22:44:14.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.633+0000 7fb4df000640 1 Processor -- start 2024-04-24T22:44:14.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.633+0000 7fb4df000640 1 -- start start 2024-04-24T22:44:14.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4df000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4d80625a0 0x7fb4d8154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4df000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d8150b70 0x7fb4d80c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4df000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4d8153c50 0x7fb4d81515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4df000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d815e230 con 0x7fb4d8150b70 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4df000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d815e0b0 con 0x7fb4d8153c50 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4df000640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d81f0450 con 0x7fb4d80625a0 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4dd200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d8150b70 0x7fb4d80c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4dd200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d8150b70 0x7fb4d80c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35780/0 (socket says 172.21.15.47:35780) 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4dd200640 1 -- 172.21.15.47:0/2930351375 learned_addr learned my addr 172.21.15.47:0/2930351375 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4ddc00640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4d80625a0 0x7fb4d8154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.634+0000 7fb4de600640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4d8153c50 0x7fb4d81515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4dd200640 1 -- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4d80625a0 msgr2=0x7fb4d8154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4dd200640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4d80625a0 0x7fb4d8154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4dd200640 1 -- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4d8153c50 msgr2=0x7fb4d81515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4dd200640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4d8153c50 0x7fb4d81515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4dd200640 1 -- 172.21.15.47:0/2930351375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d81553e0 con 0x7fb4d8150b70 2024-04-24T22:44:14.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4dd200640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d8150b70 0x7fb4d80c3800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400e870 tx=0x7fb4c400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:14.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4d5600640 1 -- 172.21.15.47:0/2930351375 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4017070 con 0x7fb4d8150b70 2024-04-24T22:44:14.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4d5600640 1 -- 172.21.15.47:0/2930351375 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c400c940 con 0x7fb4d8150b70 2024-04-24T22:44:14.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4d5600640 1 -- 172.21.15.47:0/2930351375 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c400cd30 con 0x7fb4d8150b70 2024-04-24T22:44:14.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d81efb40 con 0x7fb4d8150b70 2024-04-24T22:44:14.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.635+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d81efe00 con 0x7fb4d8150b70 2024-04-24T22:44:14.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.636+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4d8073b90 con 0x7fb4d8150b70 2024-04-24T22:44:14.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.638+0000 7fb4d5600640 1 -- 172.21.15.47:0/2930351375 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4c401a070 con 0x7fb4d8150b70 2024-04-24T22:44:14.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.639+0000 7fb4d5600640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb4b806be60 0x7fb4b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.639+0000 7fb4d5600640 1 -- 172.21.15.47:0/2930351375 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb4c408ae10 con 0x7fb4d8150b70 2024-04-24T22:44:14.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.639+0000 7fb4d5600640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] conn(0x7fb4b80741e0 0x7fb4b8076660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.639+0000 7fb4d5600640 1 -- 172.21.15.47:0/2930351375 --> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4c4015e50 con 0x7fb4b80741e0 2024-04-24T22:44:14.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.639+0000 7fb4d5600640 1 -- 172.21.15.47:0/2930351375 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb4c408d430 con 0x7fb4d8150b70 2024-04-24T22:44:14.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.639+0000 7fb4ddc00640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb4b806be60 0x7fb4b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.639+0000 7fb4de600640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] conn(0x7fb4b80741e0 0x7fb4b8076660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.640+0000 7fb4ddc00640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb4b806be60 0x7fb4b806e320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc034000 tx=0x7fb4cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:14.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.640+0000 7fb4de600640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] conn(0x7fb4b80741e0 0x7fb4b8076660 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:14.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.641+0000 7fb4d5600640 1 -- 172.21.15.47:0/2930351375 <== osd.5 v2:172.21.15.113:6800/1697310705 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4c4015e50 con 0x7fb4b80741e0 2024-04-24T22:44:14.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.655+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 --> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb4d8072fb0 con 0x7fb4b80741e0 2024-04-24T22:44:14.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.656+0000 7fb4d5600640 1 -- 172.21.15.47:0/2930351375 <== osd.5 v2:172.21.15.113:6800/1697310705 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb4d8072fb0 con 0x7fb4b80741e0 2024-04-24T22:44:14.658 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:44:14.658 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T22:44:14.658 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:44:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] conn(0x7fb4b80741e0 msgr2=0x7fb4b8076660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4df000640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:6800/1697310705,v1:172.21.15.113:6801/1697310705] conn(0x7fb4b80741e0 0x7fb4b8076660 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb4b806be60 msgr2=0x7fb4b806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4df000640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb4b806be60 0x7fb4b806e320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc034000 tx=0x7fb4cc02e000 comp rx=0 tx=0).stop 2024-04-24T22:44:14.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d8150b70 msgr2=0x7fb4d80c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4df000640 1 --2- 172.21.15.47:0/2930351375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4d8150b70 0x7fb4d80c3800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400e870 tx=0x7fb4c400ed30 comp rx=0 tx=0).stop 2024-04-24T22:44:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4de600640 1 -- 172.21.15.47:0/2930351375 reap_dead start 2024-04-24T22:44:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 shutdown_connections 2024-04-24T22:44:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 >> 172.21.15.47:0/2930351375 conn(0x7fb4d806b160 msgr2=0x7fb4d806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 shutdown_connections 2024-04-24T22:44:14.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.657+0000 7fb4df000640 1 -- 172.21.15.47:0/2930351375 wait complete. 2024-04-24T22:44:14.678 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-24T22:44:14.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.890+0000 7fe928800640 1 Processor -- start 2024-04-24T22:44:14.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.890+0000 7fe928800640 1 -- start start 2024-04-24T22:44:14.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.892+0000 7fe928800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9200c8fa0 0x7fe920163dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.892+0000 7fe928800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9200c7bb0 con 0x7fe9200c7750 2024-04-24T22:44:14.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.892+0000 7fe928800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9200c7e50 con 0x7fe920152960 2024-04-24T22:44:14.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.892+0000 7fe928800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe920158f20 con 0x7fe9200c8fa0 2024-04-24T22:44:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.892+0000 7fe925400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe9200c7750 0x7fe920072b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46714/0 (socket says 172.21.15.47:46714) 2024-04-24T22:44:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.892+0000 7fe925400640 1 -- 172.21.15.47:0/823216422 learned_addr learned my addr 172.21.15.47:0/823216422 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.892+0000 7fe926800640 1 --2- 172.21.15.47:0/823216422 >> v2:172.21.15.113:3300/0 conn(0x7fe9200c8fa0 0x7fe920163dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.892+0000 7fe924a00640 1 -- 172.21.15.47:0/823216422 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3166838083 0 0) 0x7fe9200c7e50 con 0x7fe920152960 2024-04-24T22:44:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.893+0000 7fe924a00640 1 -- 172.21.15.47:0/823216422 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe908003520 con 0x7fe920152960 2024-04-24T22:44:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.893+0000 7fe924a00640 1 -- 172.21.15.47:0/823216422 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2785187253 0 0) 0x7fe9200c7bb0 con 0x7fe9200c7750 2024-04-24T22:44:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.893+0000 7fe924a00640 1 -- 172.21.15.47:0/823216422 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9200c7e50 con 0x7fe9200c7750 2024-04-24T22:44:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.893+0000 7fe926800640 1 -- 172.21.15.47:0/823216422 >> v1:172.21.15.113:6789/0 conn(0x7fe920152960 legacy=0x7fe92015ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.893+0000 7fe926800640 1 -- 172.21.15.47:0/823216422 >> v1:172.21.15.47:6789/0 conn(0x7fe9200c7750 legacy=0x7fe920072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.893+0000 7fe926800640 1 -- 172.21.15.47:0/823216422 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9201585f0 con 0x7fe9200c8fa0 2024-04-24T22:44:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.894+0000 7fe926800640 1 --2- 172.21.15.47:0/823216422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9200c8fa0 0x7fe920163dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009d90 tx=0x7fe91c02a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=211b8012a47d221b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.894+0000 7fe924a00640 1 -- 172.21.15.47:0/823216422 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c037070 con 0x7fe9200c8fa0 2024-04-24T22:44:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.894+0000 7fe924a00640 1 -- 172.21.15.47:0/823216422 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c007740 con 0x7fe9200c8fa0 2024-04-24T22:44:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.894+0000 7fe924a00640 1 -- 172.21.15.47:0/823216422 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c007b10 con 0x7fe9200c8fa0 2024-04-24T22:44:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.895+0000 7fe928800640 1 -- 172.21.15.47:0/823216422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9200c8fa0 msgr2=0x7fe920163dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.895+0000 7fe928800640 1 --2- 172.21.15.47:0/823216422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9200c8fa0 0x7fe920163dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009d90 tx=0x7fe91c02a780 comp rx=0 tx=0).stop 2024-04-24T22:44:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.895+0000 7fe928800640 1 -- 172.21.15.47:0/823216422 shutdown_connections 2024-04-24T22:44:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.895+0000 7fe928800640 1 --2- 172.21.15.47:0/823216422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9200c8fa0 0x7fe920163dc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.895+0000 7fe928800640 1 -- 172.21.15.47:0/823216422 >> 172.21.15.47:0/823216422 conn(0x7fe92006e580 msgr2=0x7fe92006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.895+0000 7fe928800640 1 -- 172.21.15.47:0/823216422 shutdown_connections 2024-04-24T22:44:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.895+0000 7fe928800640 1 -- 172.21.15.47:0/823216422 wait complete. 2024-04-24T22:44:14.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.897+0000 7fe928800640 1 Processor -- start 2024-04-24T22:44:14.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.897+0000 7fe928800640 1 -- start start 2024-04-24T22:44:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.898+0000 7fe928800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9200c7750 0x7fe9200cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.898+0000 7fe928800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9200c8fa0 0x7fe9201f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe928800640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe920152960 0x7fe9201442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe928800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe9200c8130 con 0x7fe9200c8fa0 2024-04-24T22:44:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe928800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe9201593b0 con 0x7fe9200c7750 2024-04-24T22:44:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe928800640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe920158bb0 con 0x7fe920152960 2024-04-24T22:44:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe925400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9200c8fa0 0x7fe9201f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe925e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9200c7750 0x7fe9200cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe925400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9200c8fa0 0x7fe9201f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35784/0 (socket says 172.21.15.47:35784) 2024-04-24T22:44:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe925400640 1 -- 172.21.15.47:0/2631739531 learned_addr learned my addr 172.21.15.47:0/2631739531 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe926800640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe920152960 0x7fe9201442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe925e00640 1 -- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe920152960 msgr2=0x7fe9201442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe925e00640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe920152960 0x7fe9201442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe925e00640 1 -- 172.21.15.47:0/2631739531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9200c8fa0 msgr2=0x7fe9201f1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe925e00640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9200c8fa0 0x7fe9201f1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe925e00640 1 -- 172.21.15.47:0/2631739531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe920156c80 con 0x7fe9200c7750 2024-04-24T22:44:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.899+0000 7fe925400640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9200c8fa0 0x7fe9201f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:14.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.900+0000 7fe925e00640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9200c7750 0x7fe9200cacf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00c390 tx=0x7fe90c001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:14.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.900+0000 7fe91a000640 1 -- 172.21.15.47:0/2631739531 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c017070 con 0x7fe9200c7750 2024-04-24T22:44:14.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.900+0000 7fe91a000640 1 -- 172.21.15.47:0/2631739531 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c0052d0 con 0x7fe9200c7750 2024-04-24T22:44:14.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.900+0000 7fe91a000640 1 -- 172.21.15.47:0/2631739531 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c003830 con 0x7fe9200c7750 2024-04-24T22:44:14.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.900+0000 7fe928800640 1 -- 172.21.15.47:0/2631739531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe920156f20 con 0x7fe9200c7750 2024-04-24T22:44:14.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.900+0000 7fe928800640 1 -- 172.21.15.47:0/2631739531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9200c52f0 con 0x7fe9200c7750 2024-04-24T22:44:14.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.901+0000 7fe928800640 1 -- 172.21.15.47:0/2631739531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe920073b90 con 0x7fe9200c7750 2024-04-24T22:44:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.902+0000 7fe91a000640 1 -- 172.21.15.47:0/2631739531 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe90c01a070 con 0x7fe9200c7750 2024-04-24T22:44:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.903+0000 7fe91a000640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe90006be30 0x7fe90006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.903+0000 7fe91a000640 1 -- 172.21.15.47:0/2631739531 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe90c08a250 con 0x7fe9200c7750 2024-04-24T22:44:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.903+0000 7fe925400640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe90006be30 0x7fe90006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.904+0000 7fe925400640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe90006be30 0x7fe90006e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe92006c000 tx=0x7fe914009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:14.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:14.908+0000 7fe91a000640 1 -- 172.21.15.47:0/2631739531 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe90c058490 con 0x7fe9200c7750 2024-04-24T22:44:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.114+0000 7fe928800640 1 -- 172.21.15.47:0/2631739531 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fe9200c5780 con 0x7fe90006be30 2024-04-24T22:44:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.115+0000 7fe91a000640 1 -- 172.21.15.47:0/2631739531 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe9200c5780 con 0x7fe90006be30 2024-04-24T22:44:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 5 to deep-scrub 2024-04-24T22:44:15.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.117+0000 7fe928800640 1 -- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe90006be30 msgr2=0x7fe90006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.117+0000 7fe928800640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe90006be30 0x7fe90006e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe92006c000 tx=0x7fe914009040 comp rx=0 tx=0).stop 2024-04-24T22:44:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.117+0000 7fe928800640 1 -- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9200c7750 msgr2=0x7fe9200cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.117+0000 7fe928800640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9200c7750 0x7fe9200cacf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00c390 tx=0x7fe90c001e70 comp rx=0 tx=0).stop 2024-04-24T22:44:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.118+0000 7fe928800640 1 -- 172.21.15.47:0/2631739531 shutdown_connections 2024-04-24T22:44:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.118+0000 7fe928800640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe90006be30 0x7fe90006e2f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.118+0000 7fe928800640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe920152960 0x7fe9201442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.118+0000 7fe928800640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9200c8fa0 0x7fe9201f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.118+0000 7fe928800640 1 --2- 172.21.15.47:0/2631739531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9200c7750 0x7fe9200cacf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.118+0000 7fe928800640 1 -- 172.21.15.47:0/2631739531 >> 172.21.15.47:0/2631739531 conn(0x7fe92006e580 msgr2=0x7fe920157950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.118+0000 7fe928800640 1 -- 172.21.15.47:0/2631739531 shutdown_connections 2024-04-24T22:44:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.118+0000 7fe928800640 1 -- 172.21.15.47:0/2631739531 wait complete. 2024-04-24T22:44:15.139 INFO:tasks.ceph:Scrubbing osd.6 2024-04-24T22:44:15.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-24T22:44:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.343+0000 7f4cade00640 1 Processor -- start 2024-04-24T22:44:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.344+0000 7f4cade00640 1 -- start start 2024-04-24T22:44:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.345+0000 7f4cade00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4ca80cd010 0x7f4ca80ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.345+0000 7f4cade00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca8073680 con 0x7f4ca806b160 2024-04-24T22:44:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.345+0000 7f4cade00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca815dea0 con 0x7f4ca8154840 2024-04-24T22:44:15.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.345+0000 7f4cade00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca80cc470 con 0x7f4ca80cd010 2024-04-24T22:44:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.346+0000 7f4ca7400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4ca806b160 0x7f4ca806b590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46726/0 (socket says 172.21.15.47:46726) 2024-04-24T22:44:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.346+0000 7f4ca7400640 1 -- 172.21.15.47:0/777317712 learned_addr learned my addr 172.21.15.47:0/777317712 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.346+0000 7f4ca6a00640 1 --1- 172.21.15.47:0/777317712 >> v1:172.21.15.113:6789/0 conn(0x7f4ca8154840 0x7f4ca80cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:45342/0 (socket says 172.21.15.47:45342) 2024-04-24T22:44:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.346+0000 7f4c9e000640 1 --2- 172.21.15.47:0/777317712 >> v2:172.21.15.47:3300/0 conn(0x7f4ca80cd010 0x7f4ca80ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.346+0000 7f4ca6000640 1 -- 172.21.15.47:0/777317712 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3188879320 0 0) 0x7f4ca815dea0 con 0x7f4ca8154840 2024-04-24T22:44:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.346+0000 7f4ca6000640 1 -- 172.21.15.47:0/777317712 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c8c003520 con 0x7f4ca8154840 2024-04-24T22:44:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.346+0000 7f4c9e000640 1 -- 172.21.15.47:0/777317712 >> v1:172.21.15.113:6789/0 conn(0x7f4ca8154840 legacy=0x7f4ca80cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.346+0000 7f4c9e000640 1 -- 172.21.15.47:0/777317712 >> v1:172.21.15.47:6789/0 conn(0x7f4ca806b160 legacy=0x7f4ca806b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.346+0000 7f4c9e000640 1 -- 172.21.15.47:0/777317712 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca815e180 con 0x7f4ca80cd010 2024-04-24T22:44:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.347+0000 7f4c9e000640 1 --2- 172.21.15.47:0/777317712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca80cd010 0x7f4ca80ca820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80cc5f0 tx=0x7f4c9002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463d057e07745f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.348+0000 7f4ca6000640 1 -- 172.21.15.47:0/777317712 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90037070 con 0x7f4ca80cd010 2024-04-24T22:44:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.348+0000 7f4ca6000640 1 -- 172.21.15.47:0/777317712 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c90032640 con 0x7f4ca80cd010 2024-04-24T22:44:15.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.348+0000 7f4ca6000640 1 -- 172.21.15.47:0/777317712 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90032a10 con 0x7f4ca80cd010 2024-04-24T22:44:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.348+0000 7f4cade00640 1 -- 172.21.15.47:0/777317712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca80cd010 msgr2=0x7f4ca80ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.348+0000 7f4cade00640 1 --2- 172.21.15.47:0/777317712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca80cd010 0x7f4ca80ca820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80cc5f0 tx=0x7f4c9002a2c0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.348+0000 7f4cade00640 1 -- 172.21.15.47:0/777317712 shutdown_connections 2024-04-24T22:44:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.348+0000 7f4cade00640 1 --2- 172.21.15.47:0/777317712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca80cd010 0x7f4ca80ca820 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.348+0000 7f4cade00640 1 -- 172.21.15.47:0/777317712 >> 172.21.15.47:0/777317712 conn(0x7f4ca8067080 msgr2=0x7f4ca8072920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:15.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.348+0000 7f4cade00640 1 -- 172.21.15.47:0/777317712 shutdown_connections 2024-04-24T22:44:15.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.349+0000 7f4cade00640 1 -- 172.21.15.47:0/777317712 wait complete. 2024-04-24T22:44:15.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.350+0000 7f4cade00640 1 Processor -- start 2024-04-24T22:44:15.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.350+0000 7f4cade00640 1 -- start start 2024-04-24T22:44:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.351+0000 7f4cade00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ca806b160 0x7f4ca814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.351+0000 7f4cade00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca80cd010 0x7f4ca81f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.351+0000 7f4cade00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ca8154840 0x7f4ca81f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.351+0000 7f4cade00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca815ea20 con 0x7f4ca80cd010 2024-04-24T22:44:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.351+0000 7f4cade00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca815e740 con 0x7f4ca8154840 2024-04-24T22:44:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.351+0000 7f4cade00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca8154c20 con 0x7f4ca806b160 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4c9e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca80cd010 0x7f4ca81f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4ca6a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ca806b160 0x7f4ca814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4c9e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca80cd010 0x7f4ca81f3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35802/0 (socket says 172.21.15.47:35802) 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4c9e000640 1 -- 172.21.15.47:0/3581171272 learned_addr learned my addr 172.21.15.47:0/3581171272 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4ca7400640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ca8154840 0x7f4ca81f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4ca6a00640 1 -- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ca8154840 msgr2=0x7f4ca81f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4ca6a00640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ca8154840 0x7f4ca81f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4ca6a00640 1 -- 172.21.15.47:0/3581171272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca80cd010 msgr2=0x7f4ca81f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4ca6a00640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca80cd010 0x7f4ca81f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4ca6a00640 1 -- 172.21.15.47:0/3581171272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca8151210 con 0x7f4ca806b160 2024-04-24T22:44:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4c9e000640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca80cd010 0x7f4ca81f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.352+0000 7f4ca6a00640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ca806b160 0x7f4ca814f4f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c3f0 tx=0x7f4c98001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.353+0000 7f4c9fe00640 1 -- 172.21.15.47:0/3581171272 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c980074e0 con 0x7f4ca806b160 2024-04-24T22:44:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.353+0000 7f4c9fe00640 1 -- 172.21.15.47:0/3581171272 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c980076a0 con 0x7f4ca806b160 2024-04-24T22:44:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.353+0000 7f4c9fe00640 1 -- 172.21.15.47:0/3581171272 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98004a20 con 0x7f4ca806b160 2024-04-24T22:44:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.353+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca8157920 con 0x7f4ca806b160 2024-04-24T22:44:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.353+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca8157ee0 con 0x7f4ca806b160 2024-04-24T22:44:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.354+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ca8150fb0 con 0x7f4ca806b160 2024-04-24T22:44:15.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.355+0000 7f4c9fe00640 1 -- 172.21.15.47:0/3581171272 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c9801a020 con 0x7f4ca806b160 2024-04-24T22:44:15.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.356+0000 7f4c9fe00640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c8406be30 0x7f4c8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.356+0000 7f4c9fe00640 1 -- 172.21.15.47:0/3581171272 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f4c9808ac80 con 0x7f4ca806b160 2024-04-24T22:44:15.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.356+0000 7f4c9fe00640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] conn(0x7f4c840741b0 0x7f4c84076630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.356+0000 7f4c9fe00640 1 -- 172.21.15.47:0/3581171272 --> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c98010ec0 con 0x7f4c840741b0 2024-04-24T22:44:15.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.356+0000 7f4c9fe00640 1 -- 172.21.15.47:0/3581171272 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4c9808b030 con 0x7f4ca806b160 2024-04-24T22:44:15.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.356+0000 7f4c9e000640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c8406be30 0x7f4c8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.357+0000 7f4ca7400640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] conn(0x7f4c840741b0 0x7f4c84076630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.357+0000 7f4c9e000640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c8406be30 0x7f4c8406e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4c9002a300 tx=0x7f4c9002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:15.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.357+0000 7f4ca7400640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] conn(0x7f4c840741b0 0x7f4c84076630 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:15.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.358+0000 7f4c9fe00640 1 -- 172.21.15.47:0/3581171272 <== osd.6 v2:172.21.15.113:6808/1023357201 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c98010ec0 con 0x7f4c840741b0 2024-04-24T22:44:15.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.373+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 --> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4ca8073c00 con 0x7f4c840741b0 2024-04-24T22:44:15.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.373+0000 7f4c9fe00640 1 -- 172.21.15.47:0/3581171272 <== osd.6 v2:172.21.15.113:6808/1023357201 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4ca8073c00 con 0x7f4c840741b0 2024-04-24T22:44:15.375 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:44:15.375 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T22:44:15.375 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:44:15.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.374+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] conn(0x7f4c840741b0 msgr2=0x7f4c84076630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.374+0000 7f4cade00640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:6808/1023357201,v1:172.21.15.113:6809/1023357201] conn(0x7f4c840741b0 0x7f4c84076630 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.374+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c8406be30 msgr2=0x7f4c8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.374+0000 7f4cade00640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f4c8406be30 0x7f4c8406e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4c9002a300 tx=0x7f4c9002e000 comp rx=0 tx=0).stop 2024-04-24T22:44:15.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.374+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ca806b160 msgr2=0x7f4ca814f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.374+0000 7f4cade00640 1 --2- 172.21.15.47:0/3581171272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ca806b160 0x7f4ca814f4f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c3f0 tx=0x7f4c98001e70 comp rx=0 tx=0).stop 2024-04-24T22:44:15.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.374+0000 7f4ca7400640 1 -- 172.21.15.47:0/3581171272 reap_dead start 2024-04-24T22:44:15.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.374+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 shutdown_connections 2024-04-24T22:44:15.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.374+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 >> 172.21.15.47:0/3581171272 conn(0x7f4ca8067080 msgr2=0x7f4ca8072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:15.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.375+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 shutdown_connections 2024-04-24T22:44:15.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.375+0000 7f4cade00640 1 -- 172.21.15.47:0/3581171272 wait complete. 2024-04-24T22:44:15.394 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-24T22:44:15.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.606+0000 7f3213e00640 1 Processor -- start 2024-04-24T22:44:15.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.606+0000 7f3213e00640 1 -- start start 2024-04-24T22:44:15.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.607+0000 7f3213e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32141514f0 0x7f321406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.607+0000 7f3213e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f321406e6f0 0x7f32140c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.607+0000 7f3213e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3214071570 con 0x7f32140c6850 2024-04-24T22:44:15.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.607+0000 7f3213e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f321415b470 con 0x7f321406e6f0 2024-04-24T22:44:15.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.607+0000 7f3213e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f32140c6cc0 con 0x7f32141514f0 2024-04-24T22:44:15.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.608+0000 7f3212a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32141514f0 0x7f321406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.608+0000 7f3212000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f321406e6f0 0x7f32140c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.608+0000 7f3212a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32141514f0 0x7f321406e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:35054/0 (socket says 172.21.15.47:35054) 2024-04-24T22:44:15.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.608+0000 7f3212a00640 1 -- 172.21.15.47:0/1375379425 learned_addr learned my addr 172.21.15.47:0/1375379425 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:15.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.608+0000 7f3212000640 1 -- 172.21.15.47:0/1375379425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32141514f0 msgr2=0x7f321406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:15.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.608+0000 7f3212000640 1 --2- 172.21.15.47:0/1375379425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32141514f0 0x7f321406e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.608+0000 7f3212000640 1 -- 172.21.15.47:0/1375379425 >> v1:172.21.15.113:6789/0 conn(0x7f32140c6850 legacy=0x7f32140707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.608+0000 7f3212000640 1 -- 172.21.15.47:0/1375379425 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321415a8a0 con 0x7f321406e6f0 2024-04-24T22:44:15.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.609+0000 7f3212000640 1 --2- 172.21.15.47:0/1375379425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321406e6f0 0x7f32140c6310 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009d90 tx=0x7f31fc02c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0c51bab5656f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:15.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.610+0000 7f3211600640 1 -- 172.21.15.47:0/1375379425 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc032770 con 0x7f321406e6f0 2024-04-24T22:44:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.610+0000 7f3211600640 1 -- 172.21.15.47:0/1375379425 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31fc038070 con 0x7f321406e6f0 2024-04-24T22:44:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.610+0000 7f3211600640 1 -- 172.21.15.47:0/1375379425 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc032b30 con 0x7f321406e6f0 2024-04-24T22:44:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.610+0000 7f3213e00640 1 -- 172.21.15.47:0/1375379425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321406e6f0 msgr2=0x7f32140c6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.610+0000 7f3213e00640 1 --2- 172.21.15.47:0/1375379425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321406e6f0 0x7f32140c6310 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009d90 tx=0x7f31fc02c870 comp rx=0 tx=0).stop 2024-04-24T22:44:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.610+0000 7f3213e00640 1 -- 172.21.15.47:0/1375379425 shutdown_connections 2024-04-24T22:44:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.610+0000 7f3213e00640 1 --2- 172.21.15.47:0/1375379425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f321406e6f0 0x7f32140c6310 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.610+0000 7f3213e00640 1 --2- 172.21.15.47:0/1375379425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32141514f0 0x7f321406e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.610+0000 7f3213e00640 1 -- 172.21.15.47:0/1375379425 >> 172.21.15.47:0/1375379425 conn(0x7f32140cb900 msgr2=0x7f32140cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.610+0000 7f3213e00640 1 -- 172.21.15.47:0/1375379425 shutdown_connections 2024-04-24T22:44:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.611+0000 7f3213e00640 1 -- 172.21.15.47:0/1375379425 wait complete. 2024-04-24T22:44:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.612+0000 7f3213e00640 1 Processor -- start 2024-04-24T22:44:15.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.613+0000 7f3213e00640 1 -- start start 2024-04-24T22:44:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.613+0000 7f3213e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f321406e6f0 0x7f32140c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.613+0000 7f3213e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32140c6850 0x7f321413d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3213e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32141514f0 0x7f321413d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3213e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f321415ae20 con 0x7f32140c6850 2024-04-24T22:44:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3213e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f321415ab40 con 0x7f32141514f0 2024-04-24T22:44:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3213e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f321415b100 con 0x7f321406e6f0 2024-04-24T22:44:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3212000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32140c6850 0x7f321413d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3212a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f321406e6f0 0x7f32140c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3212000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32140c6850 0x7f321413d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35820/0 (socket says 172.21.15.47:35820) 2024-04-24T22:44:15.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3212000640 1 -- 172.21.15.47:0/2648845289 learned_addr learned my addr 172.21.15.47:0/2648845289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:15.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3213400640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32141514f0 0x7f321413d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3212a00640 1 -- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32141514f0 msgr2=0x7f321413d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3212a00640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32141514f0 0x7f321413d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3212a00640 1 -- 172.21.15.47:0/2648845289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32140c6850 msgr2=0x7f321413d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3212a00640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32140c6850 0x7f321413d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.614+0000 7f3212a00640 1 -- 172.21.15.47:0/2648845289 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321414aa50 con 0x7f321406e6f0 2024-04-24T22:44:15.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.615+0000 7f3212a00640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f321406e6f0 0x7f32140c84c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f320400e880 tx=0x7f320400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:15.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.615+0000 7f320aa00640 1 -- 172.21.15.47:0/2648845289 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204017070 con 0x7f321406e6f0 2024-04-24T22:44:15.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.615+0000 7f320aa00640 1 -- 172.21.15.47:0/2648845289 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320400c940 con 0x7f321406e6f0 2024-04-24T22:44:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.615+0000 7f3213e00640 1 -- 172.21.15.47:0/2648845289 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3214143b60 con 0x7f321406e6f0 2024-04-24T22:44:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.615+0000 7f320aa00640 1 -- 172.21.15.47:0/2648845289 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320400cd10 con 0x7f321406e6f0 2024-04-24T22:44:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.615+0000 7f3213e00640 1 -- 172.21.15.47:0/2648845289 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3214144120 con 0x7f321406e6f0 2024-04-24T22:44:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.616+0000 7f3213e00640 1 -- 172.21.15.47:0/2648845289 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3214068d50 con 0x7f321406e6f0 2024-04-24T22:44:15.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.617+0000 7f320aa00640 1 -- 172.21.15.47:0/2648845289 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f320401a070 con 0x7f321406e6f0 2024-04-24T22:44:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.618+0000 7f320aa00640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f31f006be60 0x7f31f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.618+0000 7f320aa00640 1 -- 172.21.15.47:0/2648845289 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f320408b1d0 con 0x7f321406e6f0 2024-04-24T22:44:15.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.619+0000 7f3212000640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f31f006be60 0x7f31f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:15.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.619+0000 7f3212000640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f31f006be60 0x7f31f006e320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f31fc000f80 tx=0x7f31fc032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:15.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.623+0000 7f320aa00640 1 -- 172.21.15.47:0/2648845289 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3204059410 con 0x7f321406e6f0 2024-04-24T22:44:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.833+0000 7f3213e00640 1 -- 172.21.15.47:0/2648845289 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f3214072000 con 0x7f31f006be60 2024-04-24T22:44:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.834+0000 7f320aa00640 1 -- 172.21.15.47:0/2648845289 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3214072000 con 0x7f31f006be60 2024-04-24T22:44:15.836 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 6 to deep-scrub 2024-04-24T22:44:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 -- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f31f006be60 msgr2=0x7f31f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f31f006be60 0x7f31f006e320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f31fc000f80 tx=0x7f31fc032000 comp rx=0 tx=0).stop 2024-04-24T22:44:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 -- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f321406e6f0 msgr2=0x7f32140c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f321406e6f0 0x7f32140c84c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f320400e880 tx=0x7f320400ed40 comp rx=0 tx=0).stop 2024-04-24T22:44:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 -- 172.21.15.47:0/2648845289 shutdown_connections 2024-04-24T22:44:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f31f006be60 0x7f31f006e320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32141514f0 0x7f321413d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32140c6850 0x7f321413d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 --2- 172.21.15.47:0/2648845289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f321406e6f0 0x7f32140c84c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 -- 172.21.15.47:0/2648845289 >> 172.21.15.47:0/2648845289 conn(0x7f32140cb900 msgr2=0x7f32140cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 -- 172.21.15.47:0/2648845289 shutdown_connections 2024-04-24T22:44:15.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:15.837+0000 7f3213e00640 1 -- 172.21.15.47:0/2648845289 wait complete. 2024-04-24T22:44:15.857 INFO:tasks.ceph:Scrubbing osd.7 2024-04-24T22:44:15.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-24T22:44:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.075+0000 7fb99f200640 1 Processor -- start 2024-04-24T22:44:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.075+0000 7fb99f200640 1 -- start start 2024-04-24T22:44:16.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.077+0000 7fb99f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 0x7fb9a00c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.077+0000 7fb99f200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9a01526b0 0x7fb9a00c6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.077+0000 7fb99f200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a00c7f10 con 0x7fb9a00c79e0 2024-04-24T22:44:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.077+0000 7fb99f200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a00c6b70 con 0x7fb9a01526b0 2024-04-24T22:44:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.077+0000 7fb99f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a00c8470 con 0x7fb9a01522d0 2024-04-24T22:44:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.077+0000 7fb99d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9a01526b0 0x7fb9a00c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.077+0000 7fb99d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb9a01526b0 0x7fb9a00c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45184/0 (socket says 172.21.15.47:45184) 2024-04-24T22:44:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.077+0000 7fb99d400640 1 -- 172.21.15.47:0/4253649237 learned_addr learned my addr 172.21.15.47:0/4253649237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.077+0000 7fb99de00640 1 --2- 172.21.15.47:0/4253649237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 0x7fb9a00c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.078+0000 7fb99d400640 1 -- 172.21.15.47:0/4253649237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 msgr2=0x7fb9a00c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.078+0000 7fb99d400640 1 --2- 172.21.15.47:0/4253649237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 0x7fb9a00c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.078+0000 7fb99d400640 1 -- 172.21.15.47:0/4253649237 >> v1:172.21.15.113:6789/0 conn(0x7fb9a00c79e0 legacy=0x7fb9a0158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.078+0000 7fb99d400640 1 -- 172.21.15.47:0/4253649237 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a00c8190 con 0x7fb9a01526b0 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.079+0000 7fb99d400640 1 --2- 172.21.15.47:0/4253649237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a01526b0 0x7fb9a00c6630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00c85f0 tx=0x7fb98802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebcf7dfd57a1797 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.079+0000 7fb99ca00640 1 -- 172.21.15.47:0/4253649237 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988037070 con 0x7fb9a01526b0 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.079+0000 7fb99ca00640 1 -- 172.21.15.47:0/4253649237 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb988032640 con 0x7fb9a01526b0 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.079+0000 7fb99ca00640 1 -- 172.21.15.47:0/4253649237 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988032a10 con 0x7fb9a01526b0 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.080+0000 7fb99f200640 1 -- 172.21.15.47:0/4253649237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a01526b0 msgr2=0x7fb9a00c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.080+0000 7fb99f200640 1 --2- 172.21.15.47:0/4253649237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a01526b0 0x7fb9a00c6630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00c85f0 tx=0x7fb98802a2e0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.080+0000 7fb99f200640 1 -- 172.21.15.47:0/4253649237 shutdown_connections 2024-04-24T22:44:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.080+0000 7fb99f200640 1 --2- 172.21.15.47:0/4253649237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a01526b0 0x7fb9a00c6630 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.080+0000 7fb99f200640 1 --2- 172.21.15.47:0/4253649237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 0x7fb9a00c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.080+0000 7fb99f200640 1 -- 172.21.15.47:0/4253649237 >> 172.21.15.47:0/4253649237 conn(0x7fb9a00cbc30 msgr2=0x7fb9a00cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.080+0000 7fb99f200640 1 -- 172.21.15.47:0/4253649237 shutdown_connections 2024-04-24T22:44:16.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.080+0000 7fb99f200640 1 -- 172.21.15.47:0/4253649237 wait complete. 2024-04-24T22:44:16.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.082+0000 7fb99f200640 1 Processor -- start 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.082+0000 7fb99f200640 1 -- start start 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a00c79e0 0x7fb9a0146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 0x7fb9a00c0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a01526b0 0x7fb9a00c1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a006d660 con 0x7fb9a00c79e0 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a00c5eb0 con 0x7fb9a01526b0 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a006d7e0 con 0x7fb9a01522d0 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 0x7fb9a00c0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99d400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 0x7fb9a00c0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:35828/0 (socket says 172.21.15.47:35828) 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99d400640 1 -- 172.21.15.47:0/1919769678 learned_addr learned my addr 172.21.15.47:0/1919769678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99de00640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a00c79e0 0x7fb9a0146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.083+0000 7fb99e800640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a01526b0 0x7fb9a00c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb99d400640 1 -- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a01526b0 msgr2=0x7fb9a00c1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb99d400640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a01526b0 0x7fb9a00c1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb99d400640 1 -- 172.21.15.47:0/1919769678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a00c79e0 msgr2=0x7fb9a0146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb99d400640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a00c79e0 0x7fb9a0146b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb99d400640 1 -- 172.21.15.47:0/1919769678 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a00afcf0 con 0x7fb9a01522d0 2024-04-24T22:44:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb99de00640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9a00c79e0 0x7fb9a0146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T22:44:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb99d400640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 0x7fb9a00c0f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb988000c00 tx=0x7fb9880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb996000640 1 -- 172.21.15.47:0/1919769678 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988037040 con 0x7fb9a01522d0 2024-04-24T22:44:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb996000640 1 -- 172.21.15.47:0/1919769678 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb988042070 con 0x7fb9a01522d0 2024-04-24T22:44:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb996000640 1 -- 172.21.15.47:0/1919769678 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988030930 con 0x7fb9a01522d0 2024-04-24T22:44:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.084+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a01f3520 con 0x7fb9a01522d0 2024-04-24T22:44:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.085+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01f3ae0 con 0x7fb9a01522d0 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.085+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9a0073b90 con 0x7fb9a01522d0 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.087+0000 7fb996000640 1 -- 172.21.15.47:0/1919769678 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb988030ad0 con 0x7fb9a01522d0 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.087+0000 7fb996000640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb97c06be30 0x7fb97c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.088+0000 7fb996000640 1 -- 172.21.15.47:0/1919769678 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb9880ab7a0 con 0x7fb9a01522d0 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.088+0000 7fb996000640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] conn(0x7fb97c074150 0x7fb97c0765d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.088+0000 7fb996000640 1 -- 172.21.15.47:0/1919769678 --> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb97c076ca0 con 0x7fb97c074150 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.088+0000 7fb996000640 1 -- 172.21.15.47:0/1919769678 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb9880b2050 con 0x7fb9a01522d0 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.088+0000 7fb99de00640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb97c06be30 0x7fb97c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.088+0000 7fb99e800640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] conn(0x7fb97c074150 0x7fb97c0765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.089+0000 7fb99de00640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb97c06be30 0x7fb97c06e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb990009b90 tx=0x7fb990008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.089+0000 7fb99e800640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] conn(0x7fb97c074150 0x7fb97c0765d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.090+0000 7fb996000640 1 -- 172.21.15.47:0/1919769678 <== osd.7 v2:172.21.15.113:6816/2022093233 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb97c076ca0 con 0x7fb97c074150 2024-04-24T22:44:16.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.104+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 --> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb9a00c6cf0 con 0x7fb97c074150 2024-04-24T22:44:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.105+0000 7fb996000640 1 -- 172.21.15.47:0/1919769678 <== osd.7 v2:172.21.15.113:6816/2022093233 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb9a00c6cf0 con 0x7fb97c074150 2024-04-24T22:44:16.107 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-24T22:44:16.107 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T22:44:16.107 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-24T22:44:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] conn(0x7fb97c074150 msgr2=0x7fb97c0765d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99f200640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:6816/2022093233,v1:172.21.15.113:6818/2022093233] conn(0x7fb97c074150 0x7fb97c0765d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb97c06be30 msgr2=0x7fb97c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99f200640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb97c06be30 0x7fb97c06e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb990009b90 tx=0x7fb990008040 comp rx=0 tx=0).stop 2024-04-24T22:44:16.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 msgr2=0x7fb9a00c0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99f200640 1 --2- 172.21.15.47:0/1919769678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9a01522d0 0x7fb9a00c0f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb988000c00 tx=0x7fb9880046c0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99e800640 1 -- 172.21.15.47:0/1919769678 reap_dead start 2024-04-24T22:44:16.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 shutdown_connections 2024-04-24T22:44:16.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 >> 172.21.15.47:0/1919769678 conn(0x7fb9a00cbc30 msgr2=0x7fb9a00c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:16.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 shutdown_connections 2024-04-24T22:44:16.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.106+0000 7fb99f200640 1 -- 172.21.15.47:0/1919769678 wait complete. 2024-04-24T22:44:16.125 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-24T22:44:16.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.344+0000 7f3167200640 1 Processor -- start 2024-04-24T22:44:16.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.344+0000 7f3167200640 1 -- start start 2024-04-24T22:44:16.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.345+0000 7f3167200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3168072c60 0x7f31680c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.346+0000 7f3167200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3168073040 0x7f31681411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.346+0000 7f3167200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f316815c3d0 con 0x7f316806bb90 2024-04-24T22:44:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.346+0000 7f3167200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3168158bf0 con 0x7f3168073040 2024-04-24T22:44:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.346+0000 7f3167200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f316815c6b0 con 0x7f3168072c60 2024-04-24T22:44:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.346+0000 7f3165400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3168073040 0x7f31681411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.346+0000 7f3165e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3168072c60 0x7f31680c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.346+0000 7f3166800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f316806bb90 0x7f316815b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:53240/0 (socket says 172.21.15.47:53240) 2024-04-24T22:44:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.346+0000 7f3166800640 1 -- 172.21.15.47:0/3219517234 learned_addr learned my addr 172.21.15.47:0/3219517234 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.347+0000 7f3165400640 1 -- 172.21.15.47:0/3219517234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3168072c60 msgr2=0x7f31680c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.347+0000 7f3165400640 1 --2- 172.21.15.47:0/3219517234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3168072c60 0x7f31680c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.347+0000 7f3165400640 1 -- 172.21.15.47:0/3219517234 >> v1:172.21.15.113:6789/0 conn(0x7f316806bb90 legacy=0x7f316815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.347+0000 7f3164a00640 1 -- 172.21.15.47:0/3219517234 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2580294269 0 0) 0x7f316815c3d0 con 0x7f316806bb90 2024-04-24T22:44:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.347+0000 7f3165400640 1 -- 172.21.15.47:0/3219517234 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31681585a0 con 0x7f3168073040 2024-04-24T22:44:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.347+0000 7f3165e00640 1 --2- 172.21.15.47:0/3219517234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3168072c60 0x7f31680c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.348+0000 7f3165400640 1 --2- 172.21.15.47:0/3219517234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168073040 0x7f31681411f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f31500097a0 tx=0x7f315002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe570a770f880412 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.348+0000 7f3164a00640 1 -- 172.21.15.47:0/3219517234 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150037070 con 0x7f3168073040 2024-04-24T22:44:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.348+0000 7f3164a00640 1 -- 172.21.15.47:0/3219517234 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3150032440 con 0x7f3168073040 2024-04-24T22:44:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.348+0000 7f3164a00640 1 -- 172.21.15.47:0/3219517234 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150032810 con 0x7f3168073040 2024-04-24T22:44:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.349+0000 7f3167200640 1 -- 172.21.15.47:0/3219517234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168073040 msgr2=0x7f31681411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.349+0000 7f3167200640 1 --2- 172.21.15.47:0/3219517234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168073040 0x7f31681411f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f31500097a0 tx=0x7f315002a210 comp rx=0 tx=0).stop 2024-04-24T22:44:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.349+0000 7f3167200640 1 -- 172.21.15.47:0/3219517234 shutdown_connections 2024-04-24T22:44:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.349+0000 7f3167200640 1 --2- 172.21.15.47:0/3219517234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168073040 0x7f31681411f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.349+0000 7f3167200640 1 --2- 172.21.15.47:0/3219517234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3168072c60 0x7f31680c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.349+0000 7f3167200640 1 -- 172.21.15.47:0/3219517234 >> 172.21.15.47:0/3219517234 conn(0x7f31680cb900 msgr2=0x7f31680cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.349+0000 7f3167200640 1 -- 172.21.15.47:0/3219517234 shutdown_connections 2024-04-24T22:44:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.349+0000 7f3167200640 1 -- 172.21.15.47:0/3219517234 wait complete. 2024-04-24T22:44:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.351+0000 7f3167200640 1 Processor -- start 2024-04-24T22:44:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.351+0000 7f3167200640 1 -- start start 2024-04-24T22:44:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.352+0000 7f3167200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f316806bb90 0x7f316813b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.352+0000 7f3167200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168072c60 0x7f31681f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.352+0000 7f3167200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3168073040 0x7f3168138fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.352+0000 7f3167200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f31680c32a0 con 0x7f3168073040 2024-04-24T22:44:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.352+0000 7f3167200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3168158270 con 0x7f3168072c60 2024-04-24T22:44:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.352+0000 7f3167200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f316815bf70 con 0x7f316806bb90 2024-04-24T22:44:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.352+0000 7f3165e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f316806bb90 0x7f316813b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.352+0000 7f3165400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168072c60 0x7f31681f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.352+0000 7f3166800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3168073040 0x7f3168138fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.353+0000 7f3165400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168072c60 0x7f31681f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:57034/0 (socket says 172.21.15.47:57034) 2024-04-24T22:44:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.353+0000 7f3165400640 1 -- 172.21.15.47:0/1234700786 learned_addr learned my addr 172.21.15.47:0/1234700786 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.353+0000 7f3165400640 1 -- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f316806bb90 msgr2=0x7f316813b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.353+0000 7f3165400640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f316806bb90 0x7f316813b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.353+0000 7f3165400640 1 -- 172.21.15.47:0/1234700786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3168073040 msgr2=0x7f3168138fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.353+0000 7f3165400640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3168073040 0x7f3168138fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.353+0000 7f3165400640 1 -- 172.21.15.47:0/1234700786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31680c6960 con 0x7f3168072c60 2024-04-24T22:44:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.353+0000 7f3165e00640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f316806bb90 0x7f316813b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.353+0000 7f3165400640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168072c60 0x7f31681f4820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f31500098c0 tx=0x7f31500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.354+0000 7f315e000640 1 -- 172.21.15.47:0/1234700786 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150041070 con 0x7f3168072c60 2024-04-24T22:44:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.354+0000 7f315e000640 1 -- 172.21.15.47:0/1234700786 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f315003c870 con 0x7f3168072c60 2024-04-24T22:44:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.354+0000 7f3167200640 1 -- 172.21.15.47:0/1234700786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31680c6c30 con 0x7f3168072c60 2024-04-24T22:44:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.354+0000 7f315e000640 1 -- 172.21.15.47:0/1234700786 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150037040 con 0x7f3168072c60 2024-04-24T22:44:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.354+0000 7f3167200640 1 -- 172.21.15.47:0/1234700786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316813e980 con 0x7f3168072c60 2024-04-24T22:44:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.355+0000 7f3167200640 1 -- 172.21.15.47:0/1234700786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3168156680 con 0x7f3168072c60 2024-04-24T22:44:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.356+0000 7f315e000640 1 -- 172.21.15.47:0/1234700786 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3150044070 con 0x7f3168072c60 2024-04-24T22:44:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.357+0000 7f315e000640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f314406be60 0x7f314406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.357+0000 7f315e000640 1 -- 172.21.15.47:0/1234700786 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f31500ab8f0 con 0x7f3168072c60 2024-04-24T22:44:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.357+0000 7f3165e00640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f314406be60 0x7f314406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.358+0000 7f3165e00640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f314406be60 0x7f314406e320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3158009560 tx=0x7f3158009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:16.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.362+0000 7f315e000640 1 -- 172.21.15.47:0/1234700786 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3168156680 con 0x7f3168072c60 2024-04-24T22:44:16.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.564+0000 7f3167200640 1 -- 172.21.15.47:0/1234700786 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f316813ebc0 con 0x7f314406be60 2024-04-24T22:44:16.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.566+0000 7f315e000640 1 -- 172.21.15.47:0/1234700786 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3158011e90 con 0x7f314406be60 2024-04-24T22:44:16.567 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 7 to deep-scrub 2024-04-24T22:44:16.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.568+0000 7f3167200640 1 -- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f314406be60 msgr2=0x7f314406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.568+0000 7f3167200640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f314406be60 0x7f314406e320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3158009560 tx=0x7f3158009190 comp rx=0 tx=0).stop 2024-04-24T22:44:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.568+0000 7f3167200640 1 -- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168072c60 msgr2=0x7f31681f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.568+0000 7f3167200640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168072c60 0x7f31681f4820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f31500098c0 tx=0x7f31500046c0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.568+0000 7f3167200640 1 -- 172.21.15.47:0/1234700786 shutdown_connections 2024-04-24T22:44:16.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.569+0000 7f3167200640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f314406be60 0x7f314406e320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.569+0000 7f3167200640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3168073040 0x7f3168138fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.569+0000 7f3167200640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3168072c60 0x7f31681f4820 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.569+0000 7f3167200640 1 --2- 172.21.15.47:0/1234700786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f316806bb90 0x7f316813b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.569+0000 7f3167200640 1 -- 172.21.15.47:0/1234700786 >> 172.21.15.47:0/1234700786 conn(0x7f31680cb900 msgr2=0x7f31680cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.569+0000 7f3167200640 1 -- 172.21.15.47:0/1234700786 shutdown_connections 2024-04-24T22:44:16.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.569+0000 7f3167200640 1 -- 172.21.15.47:0/1234700786 wait complete. 2024-04-24T22:44:16.592 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:44:16.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.808+0000 7f2c60e00640 1 Processor -- start 2024-04-24T22:44:16.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.808+0000 7f2c60e00640 1 -- start start 2024-04-24T22:44:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.810+0000 7f2c60e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c5c072c60 0x7f2c5c0c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.810+0000 7f2c60e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c5c06bb90 0x7f2c5c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.810+0000 7f2c60e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c5c15c3d0 con 0x7f2c5c073040 2024-04-24T22:44:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.810+0000 7f2c60e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2c5c158bf0 con 0x7f2c5c072c60 2024-04-24T22:44:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.810+0000 7f2c60e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2c5c15c6b0 con 0x7f2c5c06bb90 2024-04-24T22:44:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.810+0000 7f2c5b400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c5c072c60 0x7f2c5c0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.810+0000 7f2c5b400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c5c072c60 0x7f2c5c0c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45212/0 (socket says 172.21.15.47:45212) 2024-04-24T22:44:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.810+0000 7f2c5b400640 1 -- 172.21.15.47:0/2507942486 learned_addr learned my addr 172.21.15.47:0/2507942486 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:16.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.810+0000 7f2c5be00640 1 --2- 172.21.15.47:0/2507942486 >> v2:172.21.15.113:3300/0 conn(0x7f2c5c06bb90 0x7f2c5c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.811+0000 7f2c5b400640 1 -- 172.21.15.47:0/2507942486 >> v2:172.21.15.113:3300/0 conn(0x7f2c5c06bb90 msgr2=0x7f2c5c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:16.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.811+0000 7f2c5b400640 1 --2- 172.21.15.47:0/2507942486 >> v2:172.21.15.113:3300/0 conn(0x7f2c5c06bb90 0x7f2c5c15b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.811+0000 7f2c5b400640 1 -- 172.21.15.47:0/2507942486 >> v1:172.21.15.47:6789/0 conn(0x7f2c5c073040 legacy=0x7f2c5c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.811+0000 7f2c5b400640 1 -- 172.21.15.47:0/2507942486 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5c1585a0 con 0x7f2c5c072c60 2024-04-24T22:44:16.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.812+0000 7f2c5b400640 1 --2- 172.21.15.47:0/2507942486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c5c072c60 0x7f2c5c0c2d60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c158b60 tx=0x7f2c4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c0eec5cdf7c402 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.813+0000 7f2c5a000640 1 -- 172.21.15.47:0/2507942486 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c037070 con 0x7f2c5c072c60 2024-04-24T22:44:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.813+0000 7f2c5a000640 1 -- 172.21.15.47:0/2507942486 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4c032440 con 0x7f2c5c072c60 2024-04-24T22:44:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.813+0000 7f2c5a000640 1 -- 172.21.15.47:0/2507942486 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c032810 con 0x7f2c5c072c60 2024-04-24T22:44:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.813+0000 7f2c60e00640 1 -- 172.21.15.47:0/2507942486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c5c072c60 msgr2=0x7f2c5c0c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.813+0000 7f2c60e00640 1 --2- 172.21.15.47:0/2507942486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c5c072c60 0x7f2c5c0c2d60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c158b60 tx=0x7f2c4c02a270 comp rx=0 tx=0).stop 2024-04-24T22:44:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.813+0000 7f2c60e00640 1 -- 172.21.15.47:0/2507942486 shutdown_connections 2024-04-24T22:44:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.813+0000 7f2c60e00640 1 --2- 172.21.15.47:0/2507942486 >> v2:172.21.15.113:3300/0 conn(0x7f2c5c06bb90 0x7f2c5c15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.813+0000 7f2c60e00640 1 --2- 172.21.15.47:0/2507942486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c5c072c60 0x7f2c5c0c2d60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.813+0000 7f2c60e00640 1 -- 172.21.15.47:0/2507942486 >> 172.21.15.47:0/2507942486 conn(0x7f2c5c0cb900 msgr2=0x7f2c5c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.814+0000 7f2c60e00640 1 -- 172.21.15.47:0/2507942486 shutdown_connections 2024-04-24T22:44:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.814+0000 7f2c60e00640 1 -- 172.21.15.47:0/2507942486 wait complete. 2024-04-24T22:44:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.815+0000 7f2c60e00640 1 Processor -- start 2024-04-24T22:44:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.815+0000 7f2c60e00640 1 -- start start 2024-04-24T22:44:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.816+0000 7f2c60e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c06bb90 0x7f2c5c1522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.816+0000 7f2c60e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c5c072c60 0x7f2c5c072170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.816+0000 7f2c60e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c5c073040 0x7f2c5c13e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.816+0000 7f2c60e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c0c32a0 con 0x7f2c5c073040 2024-04-24T22:44:16.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.816+0000 7f2c60e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c158270 con 0x7f2c5c06bb90 2024-04-24T22:44:16.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.816+0000 7f2c60e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2c5c159080 con 0x7f2c5c072c60 2024-04-24T22:44:16.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.817+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c06bb90 0x7f2c5c1522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.817+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c06bb90 0x7f2c5c1522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:57054/0 (socket says 172.21.15.47:57054) 2024-04-24T22:44:16.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.817+0000 7f2c5b400640 1 -- 172.21.15.47:0/1866129082 learned_addr learned my addr 172.21.15.47:0/1866129082 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:16.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.817+0000 7f2c5aa00640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c5c072c60 0x7f2c5c072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.817+0000 7f2c5be00640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c5c073040 0x7f2c5c13e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.817+0000 7f2c5b400640 1 -- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c5c072c60 msgr2=0x7f2c5c072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.817+0000 7f2c5b400640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c5c072c60 0x7f2c5c072170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.817+0000 7f2c5b400640 1 -- 172.21.15.47:0/1866129082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c5c073040 msgr2=0x7f2c5c13e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:16.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.817+0000 7f2c5b400640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c5c073040 0x7f2c5c13e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:16.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.817+0000 7f2c5b400640 1 -- 172.21.15.47:0/1866129082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5c153fc0 con 0x7f2c5c06bb90 2024-04-24T22:44:16.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.818+0000 7f2c5b400640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c06bb90 0x7f2c5c1522e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c02a2b0 tx=0x7f2c4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:16.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.818+0000 7f2c53400640 1 -- 172.21.15.47:0/1866129082 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c037040 con 0x7f2c5c06bb90 2024-04-24T22:44:16.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.818+0000 7f2c53400640 1 -- 172.21.15.47:0/1866129082 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4c042070 con 0x7f2c5c06bb90 2024-04-24T22:44:16.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.818+0000 7f2c53400640 1 -- 172.21.15.47:0/1866129082 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c030880 con 0x7f2c5c06bb90 2024-04-24T22:44:16.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.818+0000 7f2c60e00640 1 -- 172.21.15.47:0/1866129082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5c0718e0 con 0x7f2c5c06bb90 2024-04-24T22:44:16.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.818+0000 7f2c60e00640 1 -- 172.21.15.47:0/1866129082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5c152c00 con 0x7f2c5c06bb90 2024-04-24T22:44:16.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.819+0000 7f2c60e00640 1 -- 172.21.15.47:0/1866129082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5c068d50 con 0x7f2c5c06bb90 2024-04-24T22:44:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.820+0000 7f2c53400640 1 -- 172.21.15.47:0/1866129082 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c4c030a20 con 0x7f2c5c06bb90 2024-04-24T22:44:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.821+0000 7f2c53400640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2c3c06be60 0x7f2c3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.821+0000 7f2c53400640 1 -- 172.21.15.47:0/1866129082 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f2c4c0ab720 con 0x7f2c5c06bb90 2024-04-24T22:44:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.822+0000 7f2c5aa00640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2c3c06be60 0x7f2c3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.822+0000 7f2c5aa00640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2c3c06be60 0x7f2c3c06e320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c155710 tx=0x7f2c48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:16.826+0000 7f2c53400640 1 -- 172.21.15.47:0/1866129082 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2c4c0798d0 con 0x7f2c5c06bb90 2024-04-24T22:44:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.033+0000 7f2c60e00640 1 -- 172.21.15.47:0/1866129082 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c5c04f5e0 con 0x7f2c3c06be60 2024-04-24T22:44:17.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.042+0000 7f2c53400640 1 -- 172.21.15.47:0/1866129082 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251399 (secure 0 0 0) 0x7f2c5c04f5e0 con 0x7f2c3c06be60 2024-04-24T22:44:17.045 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:17.048 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:44:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 -- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2c3c06be60 msgr2=0x7f2c3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2c3c06be60 0x7f2c3c06e320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c155710 tx=0x7f2c48009040 comp rx=0 tx=0).stop 2024-04-24T22:44:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 -- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c06bb90 msgr2=0x7f2c5c1522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c06bb90 0x7f2c5c1522e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c02a2b0 tx=0x7f2c4c0046c0 comp rx=0 tx=0).stop 2024-04-24T22:44:17.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 -- 172.21.15.47:0/1866129082 shutdown_connections 2024-04-24T22:44:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f2c3c06be60 0x7f2c3c06e320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c5c073040 0x7f2c5c13e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c5c072c60 0x7f2c5c072170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 --2- 172.21.15.47:0/1866129082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c06bb90 0x7f2c5c1522e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 -- 172.21.15.47:0/1866129082 >> 172.21.15.47:0/1866129082 conn(0x7f2c5c0cb900 msgr2=0x7f2c5c0c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:17.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 -- 172.21.15.47:0/1866129082 shutdown_connections 2024-04-24T22:44:17.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:17.048+0000 7f2c60e00640 1 -- 172.21.15.47:0/1866129082 wait complete. 2024-04-24T22:44:17.071 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-24T22:44:16.705538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":981773,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":9797,"num_write_kb":25168828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11773,"ondisk_log_size":11773,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":286744,"kb_used_data":4056,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754687976,"statfs":{"total":773094113280,"available":772800487424,"internally_reserved":0,"allocated":4153344,"data_stored":2665906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":289394227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002944"},"pg_stats":[{"pgid":"3.3f","version":"23'167","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.315698+0000","last_change":"2024-04-24T22:40:23.914828+0000","last_active":"2024-04-24T22:44:07.315698+0000","last_peered":"2024-04-24T22:44:07.315698+0000","last_clean":"2024-04-24T22:44:07.315698+0000","last_became_active":"2024-04-24T22:40:23.913848+0000","last_became_peered":"2024-04-24T22:40:23.913848+0000","last_unstale":"2024-04-24T22:44:07.315698+0000","last_undegraded":"2024-04-24T22:44:07.315698+0000","last_fullsized":"2024-04-24T22:44:07.315698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"23'152","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.331861+0000","last_change":"2024-04-24T22:39:37.697245+0000","last_active":"2024-04-24T22:44:07.331861+0000","last_peered":"2024-04-24T22:44:07.331861+0000","last_clean":"2024-04-24T22:44:07.331861+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:44:07.331861+0000","last_undegraded":"2024-04-24T22:44:07.331861+0000","last_fullsized":"2024-04-24T22:44:07.331861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'178","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.341440+0000","last_change":"2024-04-24T22:39:37.709539+0000","last_active":"2024-04-24T22:44:07.341440+0000","last_peered":"2024-04-24T22:44:07.341440+0000","last_clean":"2024-04-24T22:44:07.341440+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:44:07.341440+0000","last_undegraded":"2024-04-24T22:44:07.341440+0000","last_fullsized":"2024-04-24T22:44:07.341440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'193","reported_seq":211,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.328595+0000","last_change":"2024-04-24T22:39:37.719136+0000","last_active":"2024-04-24T22:44:07.328595+0000","last_peered":"2024-04-24T22:44:07.328595+0000","last_clean":"2024-04-24T22:44:07.328595+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:44:07.328595+0000","last_undegraded":"2024-04-24T22:44:07.328595+0000","last_fullsized":"2024-04-24T22:44:07.328595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'174","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.316719+0000","last_change":"2024-04-24T22:39:37.704671+0000","last_active":"2024-04-24T22:44:07.316719+0000","last_peered":"2024-04-24T22:44:07.316719+0000","last_clean":"2024-04-24T22:44:07.316719+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:44:07.316719+0000","last_undegraded":"2024-04-24T22:44:07.316719+0000","last_fullsized":"2024-04-24T22:44:07.316719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:24.540286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'162","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.344177+0000","last_change":"2024-04-24T22:39:37.703880+0000","last_active":"2024-04-24T22:44:07.344177+0000","last_peered":"2024-04-24T22:44:07.344177+0000","last_clean":"2024-04-24T22:44:07.344177+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:44:07.344177+0000","last_undegraded":"2024-04-24T22:44:07.344177+0000","last_fullsized":"2024-04-24T22:44:07.344177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'188","reported_seq":205,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.338808+0000","last_change":"2024-04-24T22:39:37.703539+0000","last_active":"2024-04-24T22:44:07.338808+0000","last_peered":"2024-04-24T22:44:07.338808+0000","last_clean":"2024-04-24T22:44:07.338808+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:44:07.338808+0000","last_undegraded":"2024-04-24T22:44:07.338808+0000","last_fullsized":"2024-04-24T22:44:07.338808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'204","reported_seq":222,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.328187+0000","last_change":"2024-04-24T22:39:37.713543+0000","last_active":"2024-04-24T22:44:07.328187+0000","last_peered":"2024-04-24T22:44:07.328187+0000","last_clean":"2024-04-24T22:44:07.328187+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:44:07.328187+0000","last_undegraded":"2024-04-24T22:44:07.328187+0000","last_fullsized":"2024-04-24T22:44:07.328187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'178","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.320035+0000","last_change":"2024-04-24T22:39:37.706715+0000","last_active":"2024-04-24T22:44:07.320035+0000","last_peered":"2024-04-24T22:44:07.320035+0000","last_clean":"2024-04-24T22:44:07.320035+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:44:07.320035+0000","last_undegraded":"2024-04-24T22:44:07.320035+0000","last_fullsized":"2024-04-24T22:44:07.320035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.987273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'190","reported_seq":212,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.318143+0000","last_change":"2024-04-24T22:40:23.915517+0000","last_active":"2024-04-24T22:44:07.318143+0000","last_peered":"2024-04-24T22:44:07.318143+0000","last_clean":"2024-04-24T22:44:07.318143+0000","last_became_active":"2024-04-24T22:40:23.915141+0000","last_became_peered":"2024-04-24T22:40:23.915141+0000","last_unstale":"2024-04-24T22:44:07.318143+0000","last_undegraded":"2024-04-24T22:44:07.318143+0000","last_fullsized":"2024-04-24T22:44:07.318143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:50:07.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'214","reported_seq":231,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.310371+0000","last_change":"2024-04-24T22:39:37.701069+0000","last_active":"2024-04-24T22:44:07.310371+0000","last_peered":"2024-04-24T22:44:07.310371+0000","last_clean":"2024-04-24T22:44:07.310371+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:44:07.310371+0000","last_undegraded":"2024-04-24T22:44:07.310371+0000","last_fullsized":"2024-04-24T22:44:07.310371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:14.899608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'166","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.305652+0000","last_change":"2024-04-24T22:39:37.693508+0000","last_active":"2024-04-24T22:44:07.305652+0000","last_peered":"2024-04-24T22:44:07.305652+0000","last_clean":"2024-04-24T22:44:07.305652+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:44:07.305652+0000","last_undegraded":"2024-04-24T22:44:07.305652+0000","last_fullsized":"2024-04-24T22:44:07.305652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'177","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.324130+0000","last_change":"2024-04-24T22:39:37.705157+0000","last_active":"2024-04-24T22:44:07.324130+0000","last_peered":"2024-04-24T22:44:07.324130+0000","last_clean":"2024-04-24T22:44:07.324130+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:44:07.324130+0000","last_undegraded":"2024-04-24T22:44:07.324130+0000","last_fullsized":"2024-04-24T22:44:07.324130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'172","reported_seq":190,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.343849+0000","last_change":"2024-04-24T22:39:37.698333+0000","last_active":"2024-04-24T22:44:07.343849+0000","last_peered":"2024-04-24T22:44:07.343849+0000","last_clean":"2024-04-24T22:44:07.343849+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:44:07.343849+0000","last_undegraded":"2024-04-24T22:44:07.343849+0000","last_fullsized":"2024-04-24T22:44:07.343849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'177","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.326603+0000","last_change":"2024-04-24T22:39:37.696874+0000","last_active":"2024-04-24T22:44:07.326603+0000","last_peered":"2024-04-24T22:44:07.326603+0000","last_clean":"2024-04-24T22:44:07.326603+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:44:07.326603+0000","last_undegraded":"2024-04-24T22:44:07.326603+0000","last_fullsized":"2024-04-24T22:44:07.326603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'162","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.337856+0000","last_change":"2024-04-24T22:39:37.703446+0000","last_active":"2024-04-24T22:44:07.337856+0000","last_peered":"2024-04-24T22:44:07.337856+0000","last_clean":"2024-04-24T22:44:07.337856+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:44:07.337856+0000","last_undegraded":"2024-04-24T22:44:07.337856+0000","last_fullsized":"2024-04-24T22:44:07.337856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:40.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'176","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.320802+0000","last_change":"2024-04-24T22:40:23.916129+0000","last_active":"2024-04-24T22:44:07.320802+0000","last_peered":"2024-04-24T22:44:07.320802+0000","last_clean":"2024-04-24T22:44:07.320802+0000","last_became_active":"2024-04-24T22:40:23.915441+0000","last_became_peered":"2024-04-24T22:40:23.915441+0000","last_unstale":"2024-04-24T22:44:07.320802+0000","last_undegraded":"2024-04-24T22:44:07.320802+0000","last_fullsized":"2024-04-24T22:44:07.320802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'179","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.318823+0000","last_change":"2024-04-24T22:39:37.703899+0000","last_active":"2024-04-24T22:44:07.318823+0000","last_peered":"2024-04-24T22:44:07.318823+0000","last_clean":"2024-04-24T22:44:07.318823+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:44:07.318823+0000","last_undegraded":"2024-04-24T22:44:07.318823+0000","last_fullsized":"2024-04-24T22:44:07.318823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'205","reported_seq":223,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.317690+0000","last_change":"2024-04-24T22:39:37.711019+0000","last_active":"2024-04-24T22:44:07.317690+0000","last_peered":"2024-04-24T22:44:07.317690+0000","last_clean":"2024-04-24T22:44:07.317690+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:44:07.317690+0000","last_undegraded":"2024-04-24T22:44:07.317690+0000","last_fullsized":"2024-04-24T22:44:07.317690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'179","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.306309+0000","last_change":"2024-04-24T22:39:37.696214+0000","last_active":"2024-04-24T22:44:07.306309+0000","last_peered":"2024-04-24T22:44:07.306309+0000","last_clean":"2024-04-24T22:44:07.306309+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:44:07.306309+0000","last_undegraded":"2024-04-24T22:44:07.306309+0000","last_fullsized":"2024-04-24T22:44:07.306309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:12.292673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'154","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:09.757142+0000","last_change":"2024-04-24T22:39:37.700151+0000","last_active":"2024-04-24T22:44:09.757142+0000","last_peered":"2024-04-24T22:44:09.757142+0000","last_clean":"2024-04-24T22:44:09.757142+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:44:09.757142+0000","last_undegraded":"2024-04-24T22:44:09.757142+0000","last_fullsized":"2024-04-24T22:44:09.757142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'199","reported_seq":217,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.320053+0000","last_change":"2024-04-24T22:39:37.705083+0000","last_active":"2024-04-24T22:44:07.320053+0000","last_peered":"2024-04-24T22:44:07.320053+0000","last_clean":"2024-04-24T22:44:07.320053+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:44:07.320053+0000","last_undegraded":"2024-04-24T22:44:07.320053+0000","last_fullsized":"2024-04-24T22:44:07.320053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.661397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'150","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:09.749125+0000","last_change":"2024-04-24T22:39:37.711193+0000","last_active":"2024-04-24T22:44:09.749125+0000","last_peered":"2024-04-24T22:44:09.749125+0000","last_clean":"2024-04-24T22:44:09.749125+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:44:09.749125+0000","last_undegraded":"2024-04-24T22:44:09.749125+0000","last_fullsized":"2024-04-24T22:44:09.749125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'207","reported_seq":225,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.325276+0000","last_change":"2024-04-24T22:39:37.713910+0000","last_active":"2024-04-24T22:44:07.325276+0000","last_peered":"2024-04-24T22:44:07.325276+0000","last_clean":"2024-04-24T22:44:07.325276+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:44:07.325276+0000","last_undegraded":"2024-04-24T22:44:07.325276+0000","last_fullsized":"2024-04-24T22:44:07.325276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'160","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.310852+0000","last_change":"2024-04-24T22:39:37.717191+0000","last_active":"2024-04-24T22:44:07.310852+0000","last_peered":"2024-04-24T22:44:07.310852+0000","last_clean":"2024-04-24T22:44:07.310852+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:44:07.310852+0000","last_undegraded":"2024-04-24T22:44:07.310852+0000","last_fullsized":"2024-04-24T22:44:07.310852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'217","reported_seq":235,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.315051+0000","last_change":"2024-04-24T22:39:37.702983+0000","last_active":"2024-04-24T22:44:07.315051+0000","last_peered":"2024-04-24T22:44:07.315051+0000","last_clean":"2024-04-24T22:44:07.315051+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:44:07.315051+0000","last_undegraded":"2024-04-24T22:44:07.315051+0000","last_fullsized":"2024-04-24T22:44:07.315051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:24.318987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'166","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.326816+0000","last_change":"2024-04-24T22:39:37.714111+0000","last_active":"2024-04-24T22:44:07.326816+0000","last_peered":"2024-04-24T22:44:07.326816+0000","last_clean":"2024-04-24T22:44:07.326816+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:44:07.326816+0000","last_undegraded":"2024-04-24T22:44:07.326816+0000","last_fullsized":"2024-04-24T22:44:07.326816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'152","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.321391+0000","last_change":"2024-04-24T22:39:37.714877+0000","last_active":"2024-04-24T22:44:07.321391+0000","last_peered":"2024-04-24T22:44:07.321391+0000","last_clean":"2024-04-24T22:44:07.321391+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:44:07.321391+0000","last_undegraded":"2024-04-24T22:44:07.321391+0000","last_fullsized":"2024-04-24T22:44:07.321391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'134","reported_seq":153,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.346171+0000","last_change":"2024-04-24T22:39:37.714695+0000","last_active":"2024-04-24T22:44:07.346171+0000","last_peered":"2024-04-24T22:44:07.346171+0000","last_clean":"2024-04-24T22:44:07.346171+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:44:07.346171+0000","last_undegraded":"2024-04-24T22:44:07.346171+0000","last_fullsized":"2024-04-24T22:44:07.346171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.324440+0000","last_change":"2024-04-24T22:39:37.719187+0000","last_active":"2024-04-24T22:44:07.324440+0000","last_peered":"2024-04-24T22:44:07.324440+0000","last_clean":"2024-04-24T22:44:07.324440+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:44:07.324440+0000","last_undegraded":"2024-04-24T22:44:07.324440+0000","last_fullsized":"2024-04-24T22:44:07.324440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'158","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.310975+0000","last_change":"2024-04-24T22:39:37.707283+0000","last_active":"2024-04-24T22:44:07.310975+0000","last_peered":"2024-04-24T22:44:07.310975+0000","last_clean":"2024-04-24T22:44:07.310975+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:44:07.310975+0000","last_undegraded":"2024-04-24T22:44:07.310975+0000","last_fullsized":"2024-04-24T22:44:07.310975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'180","reported_seq":198,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.338630+0000","last_change":"2024-04-24T22:39:37.697902+0000","last_active":"2024-04-24T22:44:07.338630+0000","last_peered":"2024-04-24T22:44:07.338630+0000","last_clean":"2024-04-24T22:44:07.338630+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:44:07.338630+0000","last_undegraded":"2024-04-24T22:44:07.338630+0000","last_fullsized":"2024-04-24T22:44:07.338630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:21:54.063899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'149","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.328513+0000","last_change":"2024-04-24T22:39:37.705667+0000","last_active":"2024-04-24T22:44:07.328513+0000","last_peered":"2024-04-24T22:44:07.328513+0000","last_clean":"2024-04-24T22:44:07.328513+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:44:07.328513+0000","last_undegraded":"2024-04-24T22:44:07.328513+0000","last_fullsized":"2024-04-24T22:44:07.328513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:09.705978+0000","last_change":"2024-04-24T22:39:36.707990+0000","last_active":"2024-04-24T22:42:09.705978+0000","last_peered":"2024-04-24T22:42:09.705978+0000","last_clean":"2024-04-24T22:42:09.705978+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:42:09.705978+0000","last_undegraded":"2024-04-24T22:42:09.705978+0000","last_fullsized":"2024-04-24T22:42:09.705978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.346607+0000","last_change":"2024-04-24T22:39:37.700406+0000","last_active":"2024-04-24T22:44:07.346607+0000","last_peered":"2024-04-24T22:44:07.346607+0000","last_clean":"2024-04-24T22:44:07.346607+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:44:07.346607+0000","last_undegraded":"2024-04-24T22:44:07.346607+0000","last_fullsized":"2024-04-24T22:44:07.346607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.908485+0000","last_change":"2024-04-24T22:39:36.701398+0000","last_active":"2024-04-24T22:40:23.908485+0000","last_peered":"2024-04-24T22:40:23.908485+0000","last_clean":"2024-04-24T22:40:23.908485+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:40:23.908485+0000","last_undegraded":"2024-04-24T22:40:23.908485+0000","last_fullsized":"2024-04-24T22:40:23.908485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:04.704356+0000","last_change":"2024-04-24T22:39:36.712294+0000","last_active":"2024-04-24T22:42:04.704356+0000","last_peered":"2024-04-24T22:42:04.704356+0000","last_clean":"2024-04-24T22:42:04.704356+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:42:04.704356+0000","last_undegraded":"2024-04-24T22:42:04.704356+0000","last_fullsized":"2024-04-24T22:42:04.704356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'190","reported_seq":202,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.339730+0000","last_change":"2024-04-24T22:40:23.910148+0000","last_active":"2024-04-24T22:44:07.339730+0000","last_peered":"2024-04-24T22:44:07.339730+0000","last_clean":"2024-04-24T22:44:07.339730+0000","last_became_active":"2024-04-24T22:40:23.909573+0000","last_became_peered":"2024-04-24T22:40:23.909573+0000","last_unstale":"2024-04-24T22:44:07.339730+0000","last_undegraded":"2024-04-24T22:44:07.339730+0000","last_fullsized":"2024-04-24T22:44:07.339730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906152+0000","last_change":"2024-04-24T22:39:36.698983+0000","last_active":"2024-04-24T22:40:23.906152+0000","last_peered":"2024-04-24T22:40:23.906152+0000","last_clean":"2024-04-24T22:40:23.906152+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:40:23.906152+0000","last_undegraded":"2024-04-24T22:40:23.906152+0000","last_fullsized":"2024-04-24T22:40:23.906152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'187","reported_seq":205,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.333820+0000","last_change":"2024-04-24T22:39:37.714229+0000","last_active":"2024-04-24T22:44:07.333820+0000","last_peered":"2024-04-24T22:44:07.333820+0000","last_clean":"2024-04-24T22:44:07.333820+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:44:07.333820+0000","last_undegraded":"2024-04-24T22:44:07.333820+0000","last_fullsized":"2024-04-24T22:44:07.333820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:04.691585+0000","last_change":"2024-04-24T22:39:36.705776+0000","last_active":"2024-04-24T22:42:04.691585+0000","last_peered":"2024-04-24T22:42:04.691585+0000","last_clean":"2024-04-24T22:42:04.691585+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:42:04.691585+0000","last_undegraded":"2024-04-24T22:42:04.691585+0000","last_fullsized":"2024-04-24T22:42:04.691585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'251","reported_seq":268,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.316461+0000","last_change":"2024-04-24T22:39:37.701436+0000","last_active":"2024-04-24T22:44:07.316461+0000","last_peered":"2024-04-24T22:44:07.316461+0000","last_clean":"2024-04-24T22:44:07.316461+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:44:07.316461+0000","last_undegraded":"2024-04-24T22:44:07.316461+0000","last_fullsized":"2024-04-24T22:44:07.316461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:25.093253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906331+0000","last_change":"2024-04-24T22:39:36.700237+0000","last_active":"2024-04-24T22:40:23.906331+0000","last_peered":"2024-04-24T22:40:23.906331+0000","last_clean":"2024-04-24T22:40:23.906331+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:40:23.906331+0000","last_undegraded":"2024-04-24T22:40:23.906331+0000","last_fullsized":"2024-04-24T22:40:23.906331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.339721+0000","last_change":"2024-04-24T22:39:37.703532+0000","last_active":"2024-04-24T22:44:07.339721+0000","last_peered":"2024-04-24T22:44:07.339721+0000","last_clean":"2024-04-24T22:44:07.339721+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:44:07.339721+0000","last_undegraded":"2024-04-24T22:44:07.339721+0000","last_fullsized":"2024-04-24T22:44:07.339721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'11","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:58.098338+0000","last_change":"2024-04-24T22:39:36.717477+0000","last_active":"2024-04-24T22:43:58.098338+0000","last_peered":"2024-04-24T22:43:58.098338+0000","last_clean":"2024-04-24T22:43:58.098338+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:43:58.098338+0000","last_undegraded":"2024-04-24T22:43:58.098338+0000","last_fullsized":"2024-04-24T22:43:58.098338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'168","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.317852+0000","last_change":"2024-04-24T22:39:37.699255+0000","last_active":"2024-04-24T22:44:07.317852+0000","last_peered":"2024-04-24T22:44:07.317852+0000","last_clean":"2024-04-24T22:44:07.317852+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:44:07.317852+0000","last_undegraded":"2024-04-24T22:44:07.317852+0000","last_fullsized":"2024-04-24T22:44:07.317852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'169","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.281128+0000","last_change":"2024-04-24T22:39:36.706567+0000","last_active":"2024-04-24T22:44:07.281128+0000","last_peered":"2024-04-24T22:44:07.281128+0000","last_clean":"2024-04-24T22:44:07.281128+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:44:07.281128+0000","last_undegraded":"2024-04-24T22:44:07.281128+0000","last_fullsized":"2024-04-24T22:44:07.281128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":384079,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'176","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.326735+0000","last_change":"2024-04-24T22:39:37.712064+0000","last_active":"2024-04-24T22:44:07.326735+0000","last_peered":"2024-04-24T22:44:07.326735+0000","last_clean":"2024-04-24T22:44:07.326735+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:44:07.326735+0000","last_undegraded":"2024-04-24T22:44:07.326735+0000","last_fullsized":"2024-04-24T22:44:07.326735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'15","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:12.306178+0000","last_change":"2024-04-24T22:44:12.306242+0000","last_active":"2024-04-24T22:44:12.306178+0000","last_peered":"2024-04-24T22:44:12.306178+0000","last_clean":"2024-04-24T22:44:12.306178+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:44:12.306178+0000","last_undegraded":"2024-04-24T22:44:12.306178+0000","last_fullsized":"2024-04-24T22:44:12.306178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'15","last_scrub_stamp":"2024-04-24T22:44:12.306048+0000","last_deep_scrub":"23'15","last_deep_scrub_stamp":"2024-04-24T22:44:12.306048+0000","last_clean_scrub_stamp":"2024-04-24T22:44:12.306048+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:24.572459+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'177","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:14.257959+0000","last_change":"2024-04-24T22:44:14.258000+0000","last_active":"2024-04-24T22:44:14.257959+0000","last_peered":"2024-04-24T22:44:14.257959+0000","last_clean":"2024-04-24T22:44:14.257959+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:44:14.257959+0000","last_undegraded":"2024-04-24T22:44:14.257959+0000","last_fullsized":"2024-04-24T22:44:14.257959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'177","last_scrub_stamp":"2024-04-24T22:44:14.257836+0000","last_deep_scrub":"23'177","last_deep_scrub_stamp":"2024-04-24T22:44:14.257836+0000","last_clean_scrub_stamp":"2024-04-24T22:44:14.257836+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:52.790523+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906390+0000","last_change":"2024-04-24T22:39:36.706454+0000","last_active":"2024-04-24T22:40:23.906390+0000","last_peered":"2024-04-24T22:40:23.906390+0000","last_clean":"2024-04-24T22:40:23.906390+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:40:23.906390+0000","last_undegraded":"2024-04-24T22:40:23.906390+0000","last_fullsized":"2024-04-24T22:40:23.906390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'201","reported_seq":218,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.314981+0000","last_change":"2024-04-24T22:39:37.696724+0000","last_active":"2024-04-24T22:44:07.314981+0000","last_peered":"2024-04-24T22:44:07.314981+0000","last_clean":"2024-04-24T22:44:07.314981+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:44:07.314981+0000","last_undegraded":"2024-04-24T22:44:07.314981+0000","last_fullsized":"2024-04-24T22:44:07.314981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:24.926932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.908547+0000","last_change":"2024-04-24T22:39:36.701340+0000","last_active":"2024-04-24T22:40:23.908547+0000","last_peered":"2024-04-24T22:40:23.908547+0000","last_clean":"2024-04-24T22:40:23.908547+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:40:23.908547+0000","last_undegraded":"2024-04-24T22:40:23.908547+0000","last_fullsized":"2024-04-24T22:40:23.908547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'197","reported_seq":214,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.332223+0000","last_change":"2024-04-24T22:39:37.697532+0000","last_active":"2024-04-24T22:44:07.332223+0000","last_peered":"2024-04-24T22:44:07.332223+0000","last_clean":"2024-04-24T22:44:07.332223+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:44:07.332223+0000","last_undegraded":"2024-04-24T22:44:07.332223+0000","last_fullsized":"2024-04-24T22:44:07.332223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:42.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'18","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:58.095124+0000","last_change":"2024-04-24T22:39:36.724649+0000","last_active":"2024-04-24T22:43:58.095124+0000","last_peered":"2024-04-24T22:43:58.095124+0000","last_clean":"2024-04-24T22:43:58.095124+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:43:58.095124+0000","last_undegraded":"2024-04-24T22:43:58.095124+0000","last_fullsized":"2024-04-24T22:43:58.095124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'215","reported_seq":232,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.314227+0000","last_change":"2024-04-24T22:39:37.702890+0000","last_active":"2024-04-24T22:44:07.314227+0000","last_peered":"2024-04-24T22:44:07.314227+0000","last_clean":"2024-04-24T22:44:07.314227+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:44:07.314227+0000","last_undegraded":"2024-04-24T22:44:07.314227+0000","last_fullsized":"2024-04-24T22:44:07.314227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:24.272128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:09.703051+0000","last_change":"2024-04-24T22:39:36.716044+0000","last_active":"2024-04-24T22:44:09.703051+0000","last_peered":"2024-04-24T22:44:09.703051+0000","last_clean":"2024-04-24T22:44:09.703051+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:44:09.703051+0000","last_undegraded":"2024-04-24T22:44:09.703051+0000","last_fullsized":"2024-04-24T22:44:09.703051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.313282+0000","last_change":"2024-04-24T22:39:37.702621+0000","last_active":"2024-04-24T22:44:07.313282+0000","last_peered":"2024-04-24T22:44:07.313282+0000","last_clean":"2024-04-24T22:44:07.313282+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:44:07.313282+0000","last_undegraded":"2024-04-24T22:44:07.313282+0000","last_fullsized":"2024-04-24T22:44:07.313282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:09.727867+0000","last_change":"2024-04-24T22:39:36.714272+0000","last_active":"2024-04-24T22:44:09.727867+0000","last_peered":"2024-04-24T22:44:09.727867+0000","last_clean":"2024-04-24T22:44:09.727867+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:44:09.727867+0000","last_undegraded":"2024-04-24T22:44:09.727867+0000","last_fullsized":"2024-04-24T22:44:09.727867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.320297+0000","last_change":"2024-04-24T22:39:37.703786+0000","last_active":"2024-04-24T22:44:07.320297+0000","last_peered":"2024-04-24T22:44:07.320297+0000","last_clean":"2024-04-24T22:44:07.320297+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:44:07.320297+0000","last_undegraded":"2024-04-24T22:44:07.320297+0000","last_fullsized":"2024-04-24T22:44:07.320297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:55.990656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'4","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:12.829705+0000","last_change":"2024-04-24T22:44:12.829772+0000","last_active":"2024-04-24T22:44:12.829705+0000","last_peered":"2024-04-24T22:44:12.829705+0000","last_clean":"2024-04-24T22:44:12.829705+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:44:12.829705+0000","last_undegraded":"2024-04-24T22:44:12.829705+0000","last_fullsized":"2024-04-24T22:44:12.829705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-04-24T22:44:12.829431+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-04-24T22:44:12.829431+0000","last_clean_scrub_stamp":"2024-04-24T22:44:12.829431+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:25.095842+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'165","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.328492+0000","last_change":"2024-04-24T22:39:37.715801+0000","last_active":"2024-04-24T22:44:07.328492+0000","last_peered":"2024-04-24T22:44:07.328492+0000","last_clean":"2024-04-24T22:44:07.328492+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:44:07.328492+0000","last_undegraded":"2024-04-24T22:44:07.328492+0000","last_fullsized":"2024-04-24T22:44:07.328492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:11.756494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'8","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:13.874883+0000","last_change":"2024-04-24T22:44:13.874930+0000","last_active":"2024-04-24T22:44:13.874883+0000","last_peered":"2024-04-24T22:44:13.874883+0000","last_clean":"2024-04-24T22:44:13.874883+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:44:13.874883+0000","last_undegraded":"2024-04-24T22:44:13.874883+0000","last_fullsized":"2024-04-24T22:44:13.874883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-04-24T22:44:13.874775+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-04-24T22:44:13.874775+0000","last_clean_scrub_stamp":"2024-04-24T22:44:13.874775+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:04.186004+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.318449+0000","last_change":"2024-04-24T22:39:37.706745+0000","last_active":"2024-04-24T22:44:07.318449+0000","last_peered":"2024-04-24T22:44:07.318449+0000","last_clean":"2024-04-24T22:44:07.318449+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:44:07.318449+0000","last_undegraded":"2024-04-24T22:44:07.318449+0000","last_fullsized":"2024-04-24T22:44:07.318449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:34.691489+0000","last_change":"2024-04-24T22:39:32.657965+0000","last_active":"2024-04-24T22:41:34.691489+0000","last_peered":"2024-04-24T22:41:34.691489+0000","last_clean":"2024-04-24T22:41:34.691489+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:41:34.691489+0000","last_undegraded":"2024-04-24T22:41:34.691489+0000","last_fullsized":"2024-04-24T22:41:34.691489+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:31.635804+0000","last_clean_scrub_stamp":"2024-04-24T22:39:31.635804+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:09.714153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.694937+0000","last_change":"2024-04-24T22:39:36.710381+0000","last_active":"2024-04-24T22:41:44.694937+0000","last_peered":"2024-04-24T22:41:44.694937+0000","last_clean":"2024-04-24T22:41:44.694937+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:41:44.694937+0000","last_undegraded":"2024-04-24T22:41:44.694937+0000","last_fullsized":"2024-04-24T22:41:44.694937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:18.504225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'183","reported_seq":202,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.323020+0000","last_change":"2024-04-24T22:39:37.713956+0000","last_active":"2024-04-24T22:44:07.323020+0000","last_peered":"2024-04-24T22:44:07.323020+0000","last_clean":"2024-04-24T22:44:07.323020+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:44:07.323020+0000","last_undegraded":"2024-04-24T22:44:07.323020+0000","last_fullsized":"2024-04-24T22:44:07.323020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:39.691975+0000","last_change":"2024-04-24T22:39:36.715290+0000","last_active":"2024-04-24T22:41:39.691975+0000","last_peered":"2024-04-24T22:41:39.691975+0000","last_clean":"2024-04-24T22:41:39.691975+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:41:39.691975+0000","last_undegraded":"2024-04-24T22:41:39.691975+0000","last_fullsized":"2024-04-24T22:41:39.691975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:57.545415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'165","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.338683+0000","last_change":"2024-04-24T22:39:37.695419+0000","last_active":"2024-04-24T22:44:07.338683+0000","last_peered":"2024-04-24T22:44:07.338683+0000","last_clean":"2024-04-24T22:44:07.338683+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:44:07.338683+0000","last_undegraded":"2024-04-24T22:44:07.338683+0000","last_fullsized":"2024-04-24T22:44:07.338683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'14","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:12.893018+0000","last_change":"2024-04-24T22:44:12.893090+0000","last_active":"2024-04-24T22:44:12.893018+0000","last_peered":"2024-04-24T22:44:12.893018+0000","last_clean":"2024-04-24T22:44:12.893018+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:44:12.893018+0000","last_undegraded":"2024-04-24T22:44:12.893018+0000","last_fullsized":"2024-04-24T22:44:12.893018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-04-24T22:44:12.892835+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-04-24T22:44:12.892835+0000","last_clean_scrub_stamp":"2024-04-24T22:44:12.892835+0000","objects_scrubbed":4,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:45.813715+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'165","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.330679+0000","last_change":"2024-04-24T22:39:37.703301+0000","last_active":"2024-04-24T22:44:07.330679+0000","last_peered":"2024-04-24T22:44:07.330679+0000","last_clean":"2024-04-24T22:44:07.330679+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:44:07.330679+0000","last_undegraded":"2024-04-24T22:44:07.330679+0000","last_fullsized":"2024-04-24T22:44:07.330679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'7","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:49.706469+0000","last_change":"2024-04-24T22:39:36.713486+0000","last_active":"2024-04-24T22:43:49.706469+0000","last_peered":"2024-04-24T22:43:49.706469+0000","last_clean":"2024-04-24T22:43:49.706469+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:43:49.706469+0000","last_undegraded":"2024-04-24T22:43:49.706469+0000","last_fullsized":"2024-04-24T22:43:49.706469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":987,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'177","reported_seq":196,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.341175+0000","last_change":"2024-04-24T22:39:37.705705+0000","last_active":"2024-04-24T22:44:07.341175+0000","last_peered":"2024-04-24T22:44:07.341175+0000","last_clean":"2024-04-24T22:44:07.341175+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:44:07.341175+0000","last_undegraded":"2024-04-24T22:44:07.341175+0000","last_fullsized":"2024-04-24T22:44:07.341175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:49.774890+0000","last_change":"2024-04-24T22:39:36.723125+0000","last_active":"2024-04-24T22:43:49.774890+0000","last_peered":"2024-04-24T22:43:49.774890+0000","last_clean":"2024-04-24T22:43:49.774890+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:43:49.774890+0000","last_undegraded":"2024-04-24T22:43:49.774890+0000","last_fullsized":"2024-04-24T22:43:49.774890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.326387+0000","last_change":"2024-04-24T22:39:37.713947+0000","last_active":"2024-04-24T22:44:07.326387+0000","last_peered":"2024-04-24T22:44:07.326387+0000","last_clean":"2024-04-24T22:44:07.326387+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:44:07.326387+0000","last_undegraded":"2024-04-24T22:44:07.326387+0000","last_fullsized":"2024-04-24T22:44:07.326387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:41.152397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:44.692421+0000","last_change":"2024-04-24T22:39:36.716541+0000","last_active":"2024-04-24T22:41:44.692421+0000","last_peered":"2024-04-24T22:41:44.692421+0000","last_clean":"2024-04-24T22:41:44.692421+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:41:44.692421+0000","last_undegraded":"2024-04-24T22:41:44.692421+0000","last_fullsized":"2024-04-24T22:41:44.692421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:33.366773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'162","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.326395+0000","last_change":"2024-04-24T22:39:37.708069+0000","last_active":"2024-04-24T22:44:07.326395+0000","last_peered":"2024-04-24T22:44:07.326395+0000","last_clean":"2024-04-24T22:44:07.326395+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:44:07.326395+0000","last_undegraded":"2024-04-24T22:44:07.326395+0000","last_fullsized":"2024-04-24T22:44:07.326395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'28","reported_seq":49,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.793785+0000","last_change":"2024-04-24T22:39:36.718740+0000","last_active":"2024-04-24T22:44:07.793785+0000","last_peered":"2024-04-24T22:44:07.793785+0000","last_clean":"2024-04-24T22:44:07.793785+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:44:07.793785+0000","last_undegraded":"2024-04-24T22:44:07.793785+0000","last_fullsized":"2024-04-24T22:44:07.793785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'166","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.321859+0000","last_change":"2024-04-24T22:39:37.704496+0000","last_active":"2024-04-24T22:44:07.321859+0000","last_peered":"2024-04-24T22:44:07.321859+0000","last_clean":"2024-04-24T22:44:07.321859+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:44:07.321859+0000","last_undegraded":"2024-04-24T22:44:07.321859+0000","last_fullsized":"2024-04-24T22:44:07.321859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:53.082764+0000","last_change":"2024-04-24T22:39:36.701308+0000","last_active":"2024-04-24T22:43:53.082764+0000","last_peered":"2024-04-24T22:43:53.082764+0000","last_clean":"2024-04-24T22:43:53.082764+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:43:53.082764+0000","last_undegraded":"2024-04-24T22:43:53.082764+0000","last_fullsized":"2024-04-24T22:43:53.082764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:33:33.919272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'183","reported_seq":202,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.336690+0000","last_change":"2024-04-24T22:39:37.700189+0000","last_active":"2024-04-24T22:44:07.336690+0000","last_peered":"2024-04-24T22:44:07.336690+0000","last_clean":"2024-04-24T22:44:07.336690+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:44:07.336690+0000","last_undegraded":"2024-04-24T22:44:07.336690+0000","last_fullsized":"2024-04-24T22:44:07.336690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:41:49.693021+0000","last_change":"2024-04-24T22:39:36.716460+0000","last_active":"2024-04-24T22:41:49.693021+0000","last_peered":"2024-04-24T22:41:49.693021+0000","last_clean":"2024-04-24T22:41:49.693021+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:41:49.693021+0000","last_undegraded":"2024-04-24T22:41:49.693021+0000","last_fullsized":"2024-04-24T22:41:49.693021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:14.623758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'162","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.311917+0000","last_change":"2024-04-24T22:39:37.704143+0000","last_active":"2024-04-24T22:44:07.311917+0000","last_peered":"2024-04-24T22:44:07.311917+0000","last_clean":"2024-04-24T22:44:07.311917+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:44:07.311917+0000","last_undegraded":"2024-04-24T22:44:07.311917+0000","last_fullsized":"2024-04-24T22:44:07.311917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:43:59.776523+0000","last_change":"2024-04-24T22:39:36.721540+0000","last_active":"2024-04-24T22:43:59.776523+0000","last_peered":"2024-04-24T22:43:59.776523+0000","last_clean":"2024-04-24T22:43:59.776523+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:43:59.776523+0000","last_undegraded":"2024-04-24T22:43:59.776523+0000","last_fullsized":"2024-04-24T22:43:59.776523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'173","reported_seq":192,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.334241+0000","last_change":"2024-04-24T22:39:37.714150+0000","last_active":"2024-04-24T22:44:07.334241+0000","last_peered":"2024-04-24T22:44:07.334241+0000","last_clean":"2024-04-24T22:44:07.334241+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:44:07.334241+0000","last_undegraded":"2024-04-24T22:44:07.334241+0000","last_fullsized":"2024-04-24T22:44:07.334241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:13.837292+0000","last_change":"2024-04-24T22:44:13.837432+0000","last_active":"2024-04-24T22:44:13.837292+0000","last_peered":"2024-04-24T22:44:13.837292+0000","last_clean":"2024-04-24T22:44:13.837292+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:44:13.837292+0000","last_undegraded":"2024-04-24T22:44:13.837292+0000","last_fullsized":"2024-04-24T22:44:13.837292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-04-24T22:44:13.836965+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-04-24T22:44:13.836965+0000","last_clean_scrub_stamp":"2024-04-24T22:44:13.836965+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:01.881690+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'164","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:09.749169+0000","last_change":"2024-04-24T22:39:37.715906+0000","last_active":"2024-04-24T22:44:09.749169+0000","last_peered":"2024-04-24T22:44:09.749169+0000","last_clean":"2024-04-24T22:44:09.749169+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:44:09.749169+0000","last_undegraded":"2024-04-24T22:44:09.749169+0000","last_fullsized":"2024-04-24T22:44:09.749169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:07.937008+0000","last_change":"2024-04-24T22:39:36.698212+0000","last_active":"2024-04-24T22:42:07.937008+0000","last_peered":"2024-04-24T22:42:07.937008+0000","last_clean":"2024-04-24T22:42:07.937008+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:42:07.937008+0000","last_undegraded":"2024-04-24T22:42:07.937008+0000","last_fullsized":"2024-04-24T22:42:07.937008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:49.342073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.306924+0000","last_change":"2024-04-24T22:39:37.696143+0000","last_active":"2024-04-24T22:44:07.306924+0000","last_peered":"2024-04-24T22:44:07.306924+0000","last_clean":"2024-04-24T22:44:07.306924+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:44:07.306924+0000","last_undegraded":"2024-04-24T22:44:07.306924+0000","last_fullsized":"2024-04-24T22:44:07.306924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:34.895475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'205","reported_seq":222,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.313442+0000","last_change":"2024-04-24T22:39:37.696113+0000","last_active":"2024-04-24T22:44:07.313442+0000","last_peered":"2024-04-24T22:44:07.313442+0000","last_clean":"2024-04-24T22:44:07.313442+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:44:07.313442+0000","last_undegraded":"2024-04-24T22:44:07.313442+0000","last_fullsized":"2024-04-24T22:44:07.313442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:02.618636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:40:23.906632+0000","last_change":"2024-04-24T22:39:36.699149+0000","last_active":"2024-04-24T22:40:23.906632+0000","last_peered":"2024-04-24T22:40:23.906632+0000","last_clean":"2024-04-24T22:40:23.906632+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:40:23.906632+0000","last_undegraded":"2024-04-24T22:40:23.906632+0000","last_fullsized":"2024-04-24T22:40:23.906632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:00:33.413773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'162","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.312373+0000","last_change":"2024-04-24T22:39:37.700972+0000","last_active":"2024-04-24T22:44:07.312373+0000","last_peered":"2024-04-24T22:44:07.312373+0000","last_clean":"2024-04-24T22:44:07.312373+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:44:07.312373+0000","last_undegraded":"2024-04-24T22:44:07.312373+0000","last_fullsized":"2024-04-24T22:44:07.312373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:38.401524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'15","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:42:09.694772+0000","last_change":"2024-04-24T22:39:36.698567+0000","last_active":"2024-04-24T22:42:09.694772+0000","last_peered":"2024-04-24T22:42:09.694772+0000","last_clean":"2024-04-24T22:42:09.694772+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:42:09.694772+0000","last_undegraded":"2024-04-24T22:42:09.694772+0000","last_fullsized":"2024-04-24T22:42:09.694772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:35.655296+0000","last_clean_scrub_stamp":"2024-04-24T22:39:35.655296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:31.747867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'160","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.342307+0000","last_change":"2024-04-24T22:39:37.706951+0000","last_active":"2024-04-24T22:44:07.342307+0000","last_peered":"2024-04-24T22:44:07.342307+0000","last_clean":"2024-04-24T22:44:07.342307+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:44:07.342307+0000","last_undegraded":"2024-04-24T22:44:07.342307+0000","last_fullsized":"2024-04-24T22:44:07.342307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:13.839259+0000","last_change":"2024-04-24T22:44:13.839383+0000","last_active":"2024-04-24T22:44:13.839259+0000","last_peered":"2024-04-24T22:44:13.839259+0000","last_clean":"2024-04-24T22:44:13.839259+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:44:13.839259+0000","last_undegraded":"2024-04-24T22:44:13.839259+0000","last_fullsized":"2024-04-24T22:44:13.839259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'2","last_scrub_stamp":"2024-04-24T22:44:13.839055+0000","last_deep_scrub":"21'2","last_deep_scrub_stamp":"2024-04-24T22:44:13.839055+0000","last_clean_scrub_stamp":"2024-04-24T22:44:13.839055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:40.467599+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'181","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:07.320794+0000","last_change":"2024-04-24T22:39:37.712078+0000","last_active":"2024-04-24T22:44:07.320794+0000","last_peered":"2024-04-24T22:44:07.320794+0000","last_clean":"2024-04-24T22:44:07.320794+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:44:07.320794+0000","last_undegraded":"2024-04-24T22:44:07.320794+0000","last_fullsized":"2024-04-24T22:44:07.320794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T22:39:36.666400+0000","last_clean_scrub_stamp":"2024-04-24T22:39:36.666400+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'19","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:13.296828+0000","last_change":"2024-04-24T22:44:13.296909+0000","last_active":"2024-04-24T22:44:13.296828+0000","last_peered":"2024-04-24T22:44:13.296828+0000","last_clean":"2024-04-24T22:44:13.296828+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:44:13.296828+0000","last_undegraded":"2024-04-24T22:44:13.296828+0000","last_fullsized":"2024-04-24T22:44:13.296828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-04-24T22:44:13.296668+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-04-24T22:44:13.296668+0000","last_clean_scrub_stamp":"2024-04-24T22:44:13.296668+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:33.532971+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11255,"ondisk_log_size":11255,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":391405,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":422,"num_write_kb":588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032192,"data_stored":782810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":401,"ondisk_log_size":401,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40348,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331492,"statfs":{"total":96636764160,"available":96595447808,"internally_reserved":0,"allocated":872448,"data_stored":680263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":651264,"data_stored":472664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":290816,"data_stored":92126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39756,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332084,"statfs":{"total":96636764160,"available":96596054016,"internally_reserved":0,"allocated":266240,"data_stored":89125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":659456,"data_stored":473203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39768,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332072,"statfs":{"total":96636764160,"available":96596041728,"internally_reserved":0,"allocated":278528,"data_stored":89762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":270336,"data_stored":89726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":864256,"data_stored":679037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":385408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":384869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:44:17.078 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.078 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.078 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.078 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.079 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.079 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.079 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.079 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.079 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.079 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.079 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.079 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.079 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.079 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.080 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.080 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.080 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.080 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.080 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.080 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.080 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.080 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.080 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.081 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.081 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.081 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.081 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.081 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.081 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.081 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.081 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.081 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.081 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.082 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.082 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.082 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.082 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.082 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.082 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.082 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.082 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.082 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.083 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.083 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.083 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.083 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.083 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.083 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.083 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.083 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.083 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.083 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.084 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.084 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.084 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.084 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.084 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.085 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.085 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.085 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.085 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-24T22:39:31.635804+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=31, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.085 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.085 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.085 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.085 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.085 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.085 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.086 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.086 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.086 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.086 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.086 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.086 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.086 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.086 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.086 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.087 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.087 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.087 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.087 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.087 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.087 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.087 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.087 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.087 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.088 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.088 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-24T22:39:35.655296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.088 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.088 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-24T22:39:36.666400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=39, tm_sec=36, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=22, tm_min=44, tm_sec=9, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T22:44:17.088 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-24T22:44:37.089 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T22:44:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.307+0000 7fc52f200640 1 Processor -- start 2024-04-24T22:44:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.307+0000 7fc52f200640 1 -- start start 2024-04-24T22:44:37.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.308+0000 7fc52f200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc528072c60 0x7fc5280c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52f200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc528073040 0x7fc5281411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52f200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52815c3d0 con 0x7fc52806bb90 2024-04-24T22:44:37.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52f200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc528158bf0 con 0x7fc528073040 2024-04-24T22:44:37.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52f200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc52815c6b0 con 0x7fc528072c60 2024-04-24T22:44:37.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc528073040 0x7fc5281411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc528073040 0x7fc5281411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42186/0 (socket says 172.21.15.47:42186) 2024-04-24T22:44:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52d400640 1 -- 172.21.15.47:0/375943346 learned_addr learned my addr 172.21.15.47:0/375943346 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52de00640 1 --2- 172.21.15.47:0/375943346 >> v2:172.21.15.113:3300/0 conn(0x7fc528072c60 0x7fc5280c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52d400640 1 -- 172.21.15.47:0/375943346 >> v2:172.21.15.113:3300/0 conn(0x7fc528072c60 msgr2=0x7fc5280c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52d400640 1 --2- 172.21.15.47:0/375943346 >> v2:172.21.15.113:3300/0 conn(0x7fc528072c60 0x7fc5280c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52d400640 1 -- 172.21.15.47:0/375943346 >> v1:172.21.15.113:6789/0 conn(0x7fc52806bb90 legacy=0x7fc52815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.309+0000 7fc52d400640 1 -- 172.21.15.47:0/375943346 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5281585a0 con 0x7fc528073040 2024-04-24T22:44:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.311+0000 7fc52d400640 1 --2- 172.21.15.47:0/375943346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc528073040 0x7fc5281411f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc52806b280 tx=0x7fc51802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6d104306dc6ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.311+0000 7fc52ca00640 1 -- 172.21.15.47:0/375943346 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518037070 con 0x7fc528073040 2024-04-24T22:44:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.311+0000 7fc52ca00640 1 -- 172.21.15.47:0/375943346 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc518032440 con 0x7fc528073040 2024-04-24T22:44:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.311+0000 7fc52ca00640 1 -- 172.21.15.47:0/375943346 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518032810 con 0x7fc528073040 2024-04-24T22:44:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.312+0000 7fc52f200640 1 -- 172.21.15.47:0/375943346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc528073040 msgr2=0x7fc5281411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.312+0000 7fc52f200640 1 --2- 172.21.15.47:0/375943346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc528073040 0x7fc5281411f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc52806b280 tx=0x7fc51802a270 comp rx=0 tx=0).stop 2024-04-24T22:44:37.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.312+0000 7fc52f200640 1 -- 172.21.15.47:0/375943346 shutdown_connections 2024-04-24T22:44:37.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.312+0000 7fc52f200640 1 --2- 172.21.15.47:0/375943346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc528073040 0x7fc5281411f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.312+0000 7fc52f200640 1 --2- 172.21.15.47:0/375943346 >> v2:172.21.15.113:3300/0 conn(0x7fc528072c60 0x7fc5280c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.312+0000 7fc52f200640 1 -- 172.21.15.47:0/375943346 >> 172.21.15.47:0/375943346 conn(0x7fc5280cb900 msgr2=0x7fc5280cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:37.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.312+0000 7fc52f200640 1 -- 172.21.15.47:0/375943346 shutdown_connections 2024-04-24T22:44:37.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.312+0000 7fc52f200640 1 -- 172.21.15.47:0/375943346 wait complete. 2024-04-24T22:44:37.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.314+0000 7fc52f200640 1 Processor -- start 2024-04-24T22:44:37.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.314+0000 7fc52f200640 1 -- start start 2024-04-24T22:44:37.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc52806bb90 0x7fc5281f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52f200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc528072c60 0x7fc528071db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc528073040 0x7fc5281f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc5280c32a0 con 0x7fc52806bb90 2024-04-24T22:44:37.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc528158270 con 0x7fc528073040 2024-04-24T22:44:37.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52f200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc528159080 con 0x7fc528072c60 2024-04-24T22:44:37.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc52806bb90 0x7fc5281f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc52806bb90 0x7fc5281f3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42196/0 (socket says 172.21.15.47:42196) 2024-04-24T22:44:37.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52de00640 1 -- 172.21.15.47:0/1080043284 learned_addr learned my addr 172.21.15.47:0/1080043284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:37.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52e800640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc528073040 0x7fc5281f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.315+0000 7fc52d400640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc528072c60 0x7fc528071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc52de00640 1 -- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc528072c60 msgr2=0x7fc528071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc52de00640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc528072c60 0x7fc528071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc52de00640 1 -- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc528073040 msgr2=0x7fc5281f0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc52de00640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc528073040 0x7fc5281f0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc52de00640 1 -- 172.21.15.47:0/1080043284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5281f0100 con 0x7fc52806bb90 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc52de00640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc52806bb90 0x7fc5281f3f70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc520009db0 tx=0x7fc52000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc51e000640 1 -- 172.21.15.47:0/1080043284 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520018020 con 0x7fc52806bb90 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc51e000640 1 -- 172.21.15.47:0/1080043284 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc520014480 con 0x7fc52806bb90 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc51e000640 1 -- 172.21.15.47:0/1080043284 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520014850 con 0x7fc52806bb90 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc52f200640 1 -- 172.21.15.47:0/1080043284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc528152940 con 0x7fc52806bb90 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.316+0000 7fc52f200640 1 -- 172.21.15.47:0/1080043284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc528152ee0 con 0x7fc52806bb90 2024-04-24T22:44:37.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.318+0000 7fc52f200640 1 -- 172.21.15.47:0/1080043284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc528068d50 con 0x7fc52806bb90 2024-04-24T22:44:37.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.318+0000 7fc51e000640 1 -- 172.21.15.47:0/1080043284 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc520005d10 con 0x7fc52806bb90 2024-04-24T22:44:37.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.319+0000 7fc51e000640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc50c06be30 0x7fc50c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.319+0000 7fc51e000640 1 -- 172.21.15.47:0/1080043284 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc52008b980 con 0x7fc52806bb90 2024-04-24T22:44:37.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.320+0000 7fc52d400640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc50c06be30 0x7fc50c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.320+0000 7fc52d400640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc50c06be30 0x7fc50c06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc518034000 tx=0x7fc51802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:37.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.324+0000 7fc51e000640 1 -- 172.21.15.47:0/1080043284 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc520022070 con 0x7fc52806bb90 2024-04-24T22:44:37.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.520+0000 7fc52f200640 1 -- 172.21.15.47:0/1080043284 --> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc52813ee10 con 0x7fc50c06be30 2024-04-24T22:44:37.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.528+0000 7fc51e000640 1 -- 172.21.15.47:0/1080043284 <== mgr.4105 v2:172.21.15.113:6832/2101666090 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253510 (secure 0 0 0) 0x7fc52813ee10 con 0x7fc50c06be30 2024-04-24T22:44:37.534 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:37.535 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-24T22:44:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 -- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc50c06be30 msgr2=0x7fc50c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc50c06be30 0x7fc50c06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc518034000 tx=0x7fc51802e000 comp rx=0 tx=0).stop 2024-04-24T22:44:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 -- 172.21.15.47:0/1080043284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc52806bb90 msgr2=0x7fc5281f3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc52806bb90 0x7fc5281f3f70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc520009db0 tx=0x7fc52000c490 comp rx=0 tx=0).stop 2024-04-24T22:44:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 -- 172.21.15.47:0/1080043284 shutdown_connections 2024-04-24T22:44:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fc50c06be30 0x7fc50c06e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc528073040 0x7fc5281f0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc528072c60 0x7fc528071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 --2- 172.21.15.47:0/1080043284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc52806bb90 0x7fc5281f3f70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 -- 172.21.15.47:0/1080043284 >> 172.21.15.47:0/1080043284 conn(0x7fc5280cb900 msgr2=0x7fc52813fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 -- 172.21.15.47:0/1080043284 shutdown_connections 2024-04-24T22:44:37.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.534+0000 7fc52f200640 1 -- 172.21.15.47:0/1080043284 wait complete. 2024-04-24T22:44:37.556 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-04-24T22:44:36.710654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":981773,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":9797,"num_write_kb":25168828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11846,"ondisk_log_size":11846,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290904,"kb_used_data":4056,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754683816,"statfs":{"total":773094113280,"available":772796227584,"internally_reserved":0,"allocated":4153344,"data_stored":2665906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":293654067},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002856"},"pg_stats":[{"pgid":"3.3f","version":"23'167","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:18.867834+0000","last_change":"2024-04-24T22:44:18.867834+0000","last_active":"2024-04-24T22:44:18.867834+0000","last_peered":"2024-04-24T22:44:18.867834+0000","last_clean":"2024-04-24T22:44:18.867834+0000","last_became_active":"2024-04-24T22:40:23.913848+0000","last_became_peered":"2024-04-24T22:40:23.913848+0000","last_unstale":"2024-04-24T22:44:18.867834+0000","last_undegraded":"2024-04-24T22:44:18.867834+0000","last_fullsized":"2024-04-24T22:44:18.867834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'167","last_scrub_stamp":"2024-04-24T22:44:18.867536+0000","last_deep_scrub":"23'167","last_deep_scrub_stamp":"2024-04-24T22:44:18.867536+0000","last_clean_scrub_stamp":"2024-04-24T22:44:18.867536+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:05.363301+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"23'152","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:19.895579+0000","last_change":"2024-04-24T22:44:19.895692+0000","last_active":"2024-04-24T22:44:19.895579+0000","last_peered":"2024-04-24T22:44:19.895579+0000","last_clean":"2024-04-24T22:44:19.895579+0000","last_became_active":"2024-04-24T22:39:37.696341+0000","last_became_peered":"2024-04-24T22:39:37.696341+0000","last_unstale":"2024-04-24T22:44:19.895579+0000","last_undegraded":"2024-04-24T22:44:19.895579+0000","last_fullsized":"2024-04-24T22:44:19.895579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'152","last_scrub_stamp":"2024-04-24T22:44:19.895237+0000","last_deep_scrub":"23'152","last_deep_scrub_stamp":"2024-04-24T22:44:19.895237+0000","last_clean_scrub_stamp":"2024-04-24T22:44:19.895237+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:10.592088+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'178","reported_seq":205,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:22.153625+0000","last_change":"2024-04-24T22:44:22.153697+0000","last_active":"2024-04-24T22:44:22.153625+0000","last_peered":"2024-04-24T22:44:22.153625+0000","last_clean":"2024-04-24T22:44:22.153625+0000","last_became_active":"2024-04-24T22:39:37.709231+0000","last_became_peered":"2024-04-24T22:39:37.709231+0000","last_unstale":"2024-04-24T22:44:22.153625+0000","last_undegraded":"2024-04-24T22:44:22.153625+0000","last_fullsized":"2024-04-24T22:44:22.153625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'178","last_scrub_stamp":"2024-04-24T22:44:22.153325+0000","last_deep_scrub":"23'178","last_deep_scrub_stamp":"2024-04-24T22:44:22.153325+0000","last_clean_scrub_stamp":"2024-04-24T22:44:22.153325+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:52.334457+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'193","reported_seq":220,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:29.916109+0000","last_change":"2024-04-24T22:44:29.916136+0000","last_active":"2024-04-24T22:44:29.916109+0000","last_peered":"2024-04-24T22:44:29.916109+0000","last_clean":"2024-04-24T22:44:29.916109+0000","last_became_active":"2024-04-24T22:39:37.718926+0000","last_became_peered":"2024-04-24T22:39:37.718926+0000","last_unstale":"2024-04-24T22:44:29.916109+0000","last_undegraded":"2024-04-24T22:44:29.916109+0000","last_fullsized":"2024-04-24T22:44:29.916109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'193","last_scrub_stamp":"2024-04-24T22:44:29.915985+0000","last_deep_scrub":"23'193","last_deep_scrub_stamp":"2024-04-24T22:44:29.915985+0000","last_clean_scrub_stamp":"2024-04-24T22:44:29.915985+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:45.802651+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'174","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:29.452205+0000","last_change":"2024-04-24T22:44:29.452231+0000","last_active":"2024-04-24T22:44:29.452205+0000","last_peered":"2024-04-24T22:44:29.452205+0000","last_clean":"2024-04-24T22:44:29.452205+0000","last_became_active":"2024-04-24T22:39:37.704406+0000","last_became_peered":"2024-04-24T22:39:37.704406+0000","last_unstale":"2024-04-24T22:44:29.452205+0000","last_undegraded":"2024-04-24T22:44:29.452205+0000","last_fullsized":"2024-04-24T22:44:29.452205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'174","last_scrub_stamp":"2024-04-24T22:44:29.452107+0000","last_deep_scrub":"23'174","last_deep_scrub_stamp":"2024-04-24T22:44:29.452107+0000","last_clean_scrub_stamp":"2024-04-24T22:44:29.452107+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:28.207961+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'162","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:24.852686+0000","last_change":"2024-04-24T22:44:24.852728+0000","last_active":"2024-04-24T22:44:24.852686+0000","last_peered":"2024-04-24T22:44:24.852686+0000","last_clean":"2024-04-24T22:44:24.852686+0000","last_became_active":"2024-04-24T22:39:37.703586+0000","last_became_peered":"2024-04-24T22:39:37.703586+0000","last_unstale":"2024-04-24T22:44:24.852686+0000","last_undegraded":"2024-04-24T22:44:24.852686+0000","last_fullsized":"2024-04-24T22:44:24.852686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-04-24T22:44:24.852589+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-04-24T22:44:24.852589+0000","last_clean_scrub_stamp":"2024-04-24T22:44:24.852589+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:36:03.141736+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'188","reported_seq":214,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:27.466738+0000","last_change":"2024-04-24T22:44:27.466765+0000","last_active":"2024-04-24T22:44:27.466738+0000","last_peered":"2024-04-24T22:44:27.466738+0000","last_clean":"2024-04-24T22:44:27.466738+0000","last_became_active":"2024-04-24T22:39:37.702571+0000","last_became_peered":"2024-04-24T22:39:37.702571+0000","last_unstale":"2024-04-24T22:44:27.466738+0000","last_undegraded":"2024-04-24T22:44:27.466738+0000","last_fullsized":"2024-04-24T22:44:27.466738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'188","last_scrub_stamp":"2024-04-24T22:44:27.466608+0000","last_deep_scrub":"23'188","last_deep_scrub_stamp":"2024-04-24T22:44:27.466608+0000","last_clean_scrub_stamp":"2024-04-24T22:44:27.466608+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:52.251521+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'204","reported_seq":230,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:20.711148+0000","last_change":"2024-04-24T22:44:20.711207+0000","last_active":"2024-04-24T22:44:20.711148+0000","last_peered":"2024-04-24T22:44:20.711148+0000","last_clean":"2024-04-24T22:44:20.711148+0000","last_became_active":"2024-04-24T22:39:37.713203+0000","last_became_peered":"2024-04-24T22:39:37.713203+0000","last_unstale":"2024-04-24T22:44:20.711148+0000","last_undegraded":"2024-04-24T22:44:20.711148+0000","last_fullsized":"2024-04-24T22:44:20.711148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'204","last_scrub_stamp":"2024-04-24T22:44:20.710960+0000","last_deep_scrub":"23'204","last_deep_scrub_stamp":"2024-04-24T22:44:20.710960+0000","last_clean_scrub_stamp":"2024-04-24T22:44:20.710960+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:59.045020+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'178","reported_seq":205,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:30.417483+0000","last_change":"2024-04-24T22:44:30.417509+0000","last_active":"2024-04-24T22:44:30.417483+0000","last_peered":"2024-04-24T22:44:30.417483+0000","last_clean":"2024-04-24T22:44:30.417483+0000","last_became_active":"2024-04-24T22:39:37.706360+0000","last_became_peered":"2024-04-24T22:39:37.706360+0000","last_unstale":"2024-04-24T22:44:30.417483+0000","last_undegraded":"2024-04-24T22:44:30.417483+0000","last_fullsized":"2024-04-24T22:44:30.417483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'178","last_scrub_stamp":"2024-04-24T22:44:30.417385+0000","last_deep_scrub":"23'178","last_deep_scrub_stamp":"2024-04-24T22:44:30.417385+0000","last_clean_scrub_stamp":"2024-04-24T22:44:30.417385+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:11.055898+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'190","reported_seq":222,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:26.997294+0000","last_change":"2024-04-24T22:44:26.997294+0000","last_active":"2024-04-24T22:44:26.997294+0000","last_peered":"2024-04-24T22:44:26.997294+0000","last_clean":"2024-04-24T22:44:26.997294+0000","last_became_active":"2024-04-24T22:40:23.915141+0000","last_became_peered":"2024-04-24T22:40:23.915141+0000","last_unstale":"2024-04-24T22:44:26.997294+0000","last_undegraded":"2024-04-24T22:44:26.997294+0000","last_fullsized":"2024-04-24T22:44:26.997294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'190","last_scrub_stamp":"2024-04-24T22:44:26.997161+0000","last_deep_scrub":"23'190","last_deep_scrub_stamp":"2024-04-24T22:44:26.997161+0000","last_clean_scrub_stamp":"2024-04-24T22:44:26.997161+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:30.957526+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'214","reported_seq":241,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:24.988554+0000","last_change":"2024-04-24T22:44:24.988554+0000","last_active":"2024-04-24T22:44:24.988554+0000","last_peered":"2024-04-24T22:44:24.988554+0000","last_clean":"2024-04-24T22:44:24.988554+0000","last_became_active":"2024-04-24T22:39:37.700786+0000","last_became_peered":"2024-04-24T22:39:37.700786+0000","last_unstale":"2024-04-24T22:44:24.988554+0000","last_undegraded":"2024-04-24T22:44:24.988554+0000","last_fullsized":"2024-04-24T22:44:24.988554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'214","last_scrub_stamp":"2024-04-24T22:44:24.988421+0000","last_deep_scrub":"23'214","last_deep_scrub_stamp":"2024-04-24T22:44:24.988421+0000","last_clean_scrub_stamp":"2024-04-24T22:44:24.988421+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:03.322481+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'166","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:16.428401+0000","last_change":"2024-04-24T22:44:16.428401+0000","last_active":"2024-04-24T22:44:16.428401+0000","last_peered":"2024-04-24T22:44:16.428401+0000","last_clean":"2024-04-24T22:44:16.428401+0000","last_became_active":"2024-04-24T22:39:37.693175+0000","last_became_peered":"2024-04-24T22:39:37.693175+0000","last_unstale":"2024-04-24T22:44:16.428401+0000","last_undegraded":"2024-04-24T22:44:16.428401+0000","last_fullsized":"2024-04-24T22:44:16.428401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'166","last_scrub_stamp":"2024-04-24T22:44:16.428214+0000","last_deep_scrub":"23'166","last_deep_scrub_stamp":"2024-04-24T22:44:16.428214+0000","last_clean_scrub_stamp":"2024-04-24T22:44:16.428214+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:34.100658+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'177","reported_seq":204,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:17.754894+0000","last_change":"2024-04-24T22:44:17.754894+0000","last_active":"2024-04-24T22:44:17.754894+0000","last_peered":"2024-04-24T22:44:17.754894+0000","last_clean":"2024-04-24T22:44:17.754894+0000","last_became_active":"2024-04-24T22:39:37.704386+0000","last_became_peered":"2024-04-24T22:39:37.704386+0000","last_unstale":"2024-04-24T22:44:17.754894+0000","last_undegraded":"2024-04-24T22:44:17.754894+0000","last_fullsized":"2024-04-24T22:44:17.754894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'177","last_scrub_stamp":"2024-04-24T22:44:17.754632+0000","last_deep_scrub":"23'177","last_deep_scrub_stamp":"2024-04-24T22:44:17.754632+0000","last_clean_scrub_stamp":"2024-04-24T22:44:17.754632+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:10:24.393430+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'172","reported_seq":199,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:16.253623+0000","last_change":"2024-04-24T22:44:16.253623+0000","last_active":"2024-04-24T22:44:16.253623+0000","last_peered":"2024-04-24T22:44:16.253623+0000","last_clean":"2024-04-24T22:44:16.253623+0000","last_became_active":"2024-04-24T22:39:37.697984+0000","last_became_peered":"2024-04-24T22:39:37.697984+0000","last_unstale":"2024-04-24T22:44:16.253623+0000","last_undegraded":"2024-04-24T22:44:16.253623+0000","last_fullsized":"2024-04-24T22:44:16.253623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'172","last_scrub_stamp":"2024-04-24T22:44:16.253349+0000","last_deep_scrub":"23'172","last_deep_scrub_stamp":"2024-04-24T22:44:16.253349+0000","last_clean_scrub_stamp":"2024-04-24T22:44:16.253349+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:47.803800+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'177","reported_seq":204,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:18.936674+0000","last_change":"2024-04-24T22:44:18.936674+0000","last_active":"2024-04-24T22:44:18.936674+0000","last_peered":"2024-04-24T22:44:18.936674+0000","last_clean":"2024-04-24T22:44:18.936674+0000","last_became_active":"2024-04-24T22:39:37.696528+0000","last_became_peered":"2024-04-24T22:39:37.696528+0000","last_unstale":"2024-04-24T22:44:18.936674+0000","last_undegraded":"2024-04-24T22:44:18.936674+0000","last_fullsized":"2024-04-24T22:44:18.936674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'177","last_scrub_stamp":"2024-04-24T22:44:18.936385+0000","last_deep_scrub":"23'177","last_deep_scrub_stamp":"2024-04-24T22:44:18.936385+0000","last_clean_scrub_stamp":"2024-04-24T22:44:18.936385+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:27.701644+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'162","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:23.836303+0000","last_change":"2024-04-24T22:44:23.836332+0000","last_active":"2024-04-24T22:44:23.836303+0000","last_peered":"2024-04-24T22:44:23.836303+0000","last_clean":"2024-04-24T22:44:23.836303+0000","last_became_active":"2024-04-24T22:39:37.703175+0000","last_became_peered":"2024-04-24T22:39:37.703175+0000","last_unstale":"2024-04-24T22:44:23.836303+0000","last_undegraded":"2024-04-24T22:44:23.836303+0000","last_fullsized":"2024-04-24T22:44:23.836303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-04-24T22:44:23.836211+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-04-24T22:44:23.836211+0000","last_clean_scrub_stamp":"2024-04-24T22:44:23.836211+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:15.110050+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'176","reported_seq":196,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:19.898009+0000","last_change":"2024-04-24T22:44:19.898072+0000","last_active":"2024-04-24T22:44:19.898009+0000","last_peered":"2024-04-24T22:44:19.898009+0000","last_clean":"2024-04-24T22:44:19.898009+0000","last_became_active":"2024-04-24T22:40:23.915441+0000","last_became_peered":"2024-04-24T22:40:23.915441+0000","last_unstale":"2024-04-24T22:44:19.898009+0000","last_undegraded":"2024-04-24T22:44:19.898009+0000","last_fullsized":"2024-04-24T22:44:19.898009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'176","last_scrub_stamp":"2024-04-24T22:44:19.897813+0000","last_deep_scrub":"23'176","last_deep_scrub_stamp":"2024-04-24T22:44:19.897813+0000","last_clean_scrub_stamp":"2024-04-24T22:44:19.897813+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:46.526356+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'179","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:16.895948+0000","last_change":"2024-04-24T22:44:16.895948+0000","last_active":"2024-04-24T22:44:16.895948+0000","last_peered":"2024-04-24T22:44:16.895948+0000","last_clean":"2024-04-24T22:44:16.895948+0000","last_became_active":"2024-04-24T22:39:37.703014+0000","last_became_peered":"2024-04-24T22:39:37.703014+0000","last_unstale":"2024-04-24T22:44:16.895948+0000","last_undegraded":"2024-04-24T22:44:16.895948+0000","last_fullsized":"2024-04-24T22:44:16.895948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'179","last_scrub_stamp":"2024-04-24T22:44:16.895787+0000","last_deep_scrub":"23'179","last_deep_scrub_stamp":"2024-04-24T22:44:16.895787+0000","last_clean_scrub_stamp":"2024-04-24T22:44:16.895787+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:11.395408+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'205","reported_seq":232,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:32.421499+0000","last_change":"2024-04-24T22:44:32.421526+0000","last_active":"2024-04-24T22:44:32.421499+0000","last_peered":"2024-04-24T22:44:32.421499+0000","last_clean":"2024-04-24T22:44:32.421499+0000","last_became_active":"2024-04-24T22:39:37.710749+0000","last_became_peered":"2024-04-24T22:39:37.710749+0000","last_unstale":"2024-04-24T22:44:32.421499+0000","last_undegraded":"2024-04-24T22:44:32.421499+0000","last_fullsized":"2024-04-24T22:44:32.421499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'205","last_scrub_stamp":"2024-04-24T22:44:32.421405+0000","last_deep_scrub":"23'205","last_deep_scrub_stamp":"2024-04-24T22:44:32.421405+0000","last_clean_scrub_stamp":"2024-04-24T22:44:32.421405+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:20.051889+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'179","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:21.945479+0000","last_change":"2024-04-24T22:44:21.945504+0000","last_active":"2024-04-24T22:44:21.945479+0000","last_peered":"2024-04-24T22:44:21.945479+0000","last_clean":"2024-04-24T22:44:21.945479+0000","last_became_active":"2024-04-24T22:39:37.695912+0000","last_became_peered":"2024-04-24T22:39:37.695912+0000","last_unstale":"2024-04-24T22:44:21.945479+0000","last_undegraded":"2024-04-24T22:44:21.945479+0000","last_fullsized":"2024-04-24T22:44:21.945479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'179","last_scrub_stamp":"2024-04-24T22:44:21.945388+0000","last_deep_scrub":"23'179","last_deep_scrub_stamp":"2024-04-24T22:44:21.945388+0000","last_clean_scrub_stamp":"2024-04-24T22:44:21.945388+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:10:28.584186+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'154","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:17.240342+0000","last_change":"2024-04-24T22:44:17.240405+0000","last_active":"2024-04-24T22:44:17.240342+0000","last_peered":"2024-04-24T22:44:17.240342+0000","last_clean":"2024-04-24T22:44:17.240342+0000","last_became_active":"2024-04-24T22:39:37.699464+0000","last_became_peered":"2024-04-24T22:39:37.699464+0000","last_unstale":"2024-04-24T22:44:17.240342+0000","last_undegraded":"2024-04-24T22:44:17.240342+0000","last_fullsized":"2024-04-24T22:44:17.240342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'154","last_scrub_stamp":"2024-04-24T22:44:17.240135+0000","last_deep_scrub":"23'154","last_deep_scrub_stamp":"2024-04-24T22:44:17.240135+0000","last_clean_scrub_stamp":"2024-04-24T22:44:17.240135+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:57.938634+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'199","reported_seq":226,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:25.887258+0000","last_change":"2024-04-24T22:44:25.887310+0000","last_active":"2024-04-24T22:44:25.887258+0000","last_peered":"2024-04-24T22:44:25.887258+0000","last_clean":"2024-04-24T22:44:25.887258+0000","last_became_active":"2024-04-24T22:39:37.704788+0000","last_became_peered":"2024-04-24T22:39:37.704788+0000","last_unstale":"2024-04-24T22:44:25.887258+0000","last_undegraded":"2024-04-24T22:44:25.887258+0000","last_fullsized":"2024-04-24T22:44:25.887258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'199","last_scrub_stamp":"2024-04-24T22:44:25.887140+0000","last_deep_scrub":"23'199","last_deep_scrub_stamp":"2024-04-24T22:44:25.887140+0000","last_clean_scrub_stamp":"2024-04-24T22:44:25.887140+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:50.672053+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'150","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:18.709628+0000","last_change":"2024-04-24T22:44:18.709785+0000","last_active":"2024-04-24T22:44:18.709628+0000","last_peered":"2024-04-24T22:44:18.709628+0000","last_clean":"2024-04-24T22:44:18.709628+0000","last_became_active":"2024-04-24T22:39:37.705742+0000","last_became_peered":"2024-04-24T22:39:37.705742+0000","last_unstale":"2024-04-24T22:44:18.709628+0000","last_undegraded":"2024-04-24T22:44:18.709628+0000","last_fullsized":"2024-04-24T22:44:18.709628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'150","last_scrub_stamp":"2024-04-24T22:44:18.709355+0000","last_deep_scrub":"23'150","last_deep_scrub_stamp":"2024-04-24T22:44:18.709355+0000","last_clean_scrub_stamp":"2024-04-24T22:44:18.709355+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:57.607946+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'207","reported_seq":234,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:26.882473+0000","last_change":"2024-04-24T22:44:26.882502+0000","last_active":"2024-04-24T22:44:26.882473+0000","last_peered":"2024-04-24T22:44:26.882473+0000","last_clean":"2024-04-24T22:44:26.882473+0000","last_became_active":"2024-04-24T22:39:37.713656+0000","last_became_peered":"2024-04-24T22:39:37.713656+0000","last_unstale":"2024-04-24T22:44:26.882473+0000","last_undegraded":"2024-04-24T22:44:26.882473+0000","last_fullsized":"2024-04-24T22:44:26.882473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'207","last_scrub_stamp":"2024-04-24T22:44:26.882365+0000","last_deep_scrub":"23'207","last_deep_scrub_stamp":"2024-04-24T22:44:26.882365+0000","last_clean_scrub_stamp":"2024-04-24T22:44:26.882365+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:31.778376+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'160","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:25.609863+0000","last_change":"2024-04-24T22:44:25.609929+0000","last_active":"2024-04-24T22:44:25.609863+0000","last_peered":"2024-04-24T22:44:25.609863+0000","last_clean":"2024-04-24T22:44:25.609863+0000","last_became_active":"2024-04-24T22:39:37.716763+0000","last_became_peered":"2024-04-24T22:39:37.716763+0000","last_unstale":"2024-04-24T22:44:25.609863+0000","last_undegraded":"2024-04-24T22:44:25.609863+0000","last_fullsized":"2024-04-24T22:44:25.609863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'160","last_scrub_stamp":"2024-04-24T22:44:25.609672+0000","last_deep_scrub":"23'160","last_deep_scrub_stamp":"2024-04-24T22:44:25.609672+0000","last_clean_scrub_stamp":"2024-04-24T22:44:25.609672+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:37.119814+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'217","reported_seq":244,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:26.010667+0000","last_change":"2024-04-24T22:44:26.010694+0000","last_active":"2024-04-24T22:44:26.010667+0000","last_peered":"2024-04-24T22:44:26.010667+0000","last_clean":"2024-04-24T22:44:26.010667+0000","last_became_active":"2024-04-24T22:39:37.702696+0000","last_became_peered":"2024-04-24T22:39:37.702696+0000","last_unstale":"2024-04-24T22:44:26.010667+0000","last_undegraded":"2024-04-24T22:44:26.010667+0000","last_fullsized":"2024-04-24T22:44:26.010667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'217","last_scrub_stamp":"2024-04-24T22:44:26.010573+0000","last_deep_scrub":"23'217","last_deep_scrub_stamp":"2024-04-24T22:44:26.010573+0000","last_clean_scrub_stamp":"2024-04-24T22:44:26.010573+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:05.948148+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'166","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:22.941086+0000","last_change":"2024-04-24T22:44:22.941146+0000","last_active":"2024-04-24T22:44:22.941086+0000","last_peered":"2024-04-24T22:44:22.941086+0000","last_clean":"2024-04-24T22:44:22.941086+0000","last_became_active":"2024-04-24T22:39:37.713867+0000","last_became_peered":"2024-04-24T22:39:37.713867+0000","last_unstale":"2024-04-24T22:44:22.941086+0000","last_undegraded":"2024-04-24T22:44:22.941086+0000","last_fullsized":"2024-04-24T22:44:22.941086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'166","last_scrub_stamp":"2024-04-24T22:44:22.940899+0000","last_deep_scrub":"23'166","last_deep_scrub_stamp":"2024-04-24T22:44:22.940899+0000","last_clean_scrub_stamp":"2024-04-24T22:44:22.940899+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:32:53.122031+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'152","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:23.639722+0000","last_change":"2024-04-24T22:44:23.639780+0000","last_active":"2024-04-24T22:44:23.639722+0000","last_peered":"2024-04-24T22:44:23.639722+0000","last_clean":"2024-04-24T22:44:23.639722+0000","last_became_active":"2024-04-24T22:39:37.714623+0000","last_became_peered":"2024-04-24T22:39:37.714623+0000","last_unstale":"2024-04-24T22:44:23.639722+0000","last_undegraded":"2024-04-24T22:44:23.639722+0000","last_fullsized":"2024-04-24T22:44:23.639722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'152","last_scrub_stamp":"2024-04-24T22:44:23.639503+0000","last_deep_scrub":"23'152","last_deep_scrub_stamp":"2024-04-24T22:44:23.639503+0000","last_clean_scrub_stamp":"2024-04-24T22:44:23.639503+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:01.595834+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'134","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:23.110858+0000","last_change":"2024-04-24T22:44:23.110918+0000","last_active":"2024-04-24T22:44:23.110858+0000","last_peered":"2024-04-24T22:44:23.110858+0000","last_clean":"2024-04-24T22:44:23.110858+0000","last_became_active":"2024-04-24T22:39:37.714370+0000","last_became_peered":"2024-04-24T22:39:37.714370+0000","last_unstale":"2024-04-24T22:44:23.110858+0000","last_undegraded":"2024-04-24T22:44:23.110858+0000","last_fullsized":"2024-04-24T22:44:23.110858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'134","last_scrub_stamp":"2024-04-24T22:44:23.110640+0000","last_deep_scrub":"23'134","last_deep_scrub_stamp":"2024-04-24T22:44:23.110640+0000","last_clean_scrub_stamp":"2024-04-24T22:44:23.110640+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:10.200283+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'153","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:30.921333+0000","last_change":"2024-04-24T22:44:30.921357+0000","last_active":"2024-04-24T22:44:30.921333+0000","last_peered":"2024-04-24T22:44:30.921333+0000","last_clean":"2024-04-24T22:44:30.921333+0000","last_became_active":"2024-04-24T22:39:37.718921+0000","last_became_peered":"2024-04-24T22:39:37.718921+0000","last_unstale":"2024-04-24T22:44:30.921333+0000","last_undegraded":"2024-04-24T22:44:30.921333+0000","last_fullsized":"2024-04-24T22:44:30.921333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'153","last_scrub_stamp":"2024-04-24T22:44:30.921230+0000","last_deep_scrub":"23'153","last_deep_scrub_stamp":"2024-04-24T22:44:30.921230+0000","last_clean_scrub_stamp":"2024-04-24T22:44:30.921230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:45.554149+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'158","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:17.889466+0000","last_change":"2024-04-24T22:44:17.889643+0000","last_active":"2024-04-24T22:44:17.889466+0000","last_peered":"2024-04-24T22:44:17.889466+0000","last_clean":"2024-04-24T22:44:17.889466+0000","last_became_active":"2024-04-24T22:39:37.706972+0000","last_became_peered":"2024-04-24T22:39:37.706972+0000","last_unstale":"2024-04-24T22:44:17.889466+0000","last_undegraded":"2024-04-24T22:44:17.889466+0000","last_fullsized":"2024-04-24T22:44:17.889466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'158","last_scrub_stamp":"2024-04-24T22:44:17.889249+0000","last_deep_scrub":"23'158","last_deep_scrub_stamp":"2024-04-24T22:44:17.889249+0000","last_clean_scrub_stamp":"2024-04-24T22:44:17.889249+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:38.125552+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'180","reported_seq":207,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:23.968169+0000","last_change":"2024-04-24T22:44:23.968208+0000","last_active":"2024-04-24T22:44:23.968169+0000","last_peered":"2024-04-24T22:44:23.968169+0000","last_clean":"2024-04-24T22:44:23.968169+0000","last_became_active":"2024-04-24T22:39:37.697597+0000","last_became_peered":"2024-04-24T22:39:37.697597+0000","last_unstale":"2024-04-24T22:44:23.968169+0000","last_undegraded":"2024-04-24T22:44:23.968169+0000","last_fullsized":"2024-04-24T22:44:23.968169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'180","last_scrub_stamp":"2024-04-24T22:44:23.968068+0000","last_deep_scrub":"23'180","last_deep_scrub_stamp":"2024-04-24T22:44:23.968068+0000","last_clean_scrub_stamp":"2024-04-24T22:44:23.968068+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:03.837991+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'149","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:19.492993+0000","last_change":"2024-04-24T22:44:19.493018+0000","last_active":"2024-04-24T22:44:19.492993+0000","last_peered":"2024-04-24T22:44:19.492993+0000","last_clean":"2024-04-24T22:44:19.492993+0000","last_became_active":"2024-04-24T22:39:37.705404+0000","last_became_peered":"2024-04-24T22:39:37.705404+0000","last_unstale":"2024-04-24T22:44:19.492993+0000","last_undegraded":"2024-04-24T22:44:19.492993+0000","last_fullsized":"2024-04-24T22:44:19.492993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'149","last_scrub_stamp":"2024-04-24T22:44:19.492904+0000","last_deep_scrub":"23'149","last_deep_scrub_stamp":"2024-04-24T22:44:19.492904+0000","last_clean_scrub_stamp":"2024-04-24T22:44:19.492904+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:05:28.863245+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'7","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:14.811730+0000","last_change":"2024-04-24T22:44:14.811780+0000","last_active":"2024-04-24T22:44:14.811730+0000","last_peered":"2024-04-24T22:44:14.811730+0000","last_clean":"2024-04-24T22:44:14.811730+0000","last_became_active":"2024-04-24T22:39:36.707676+0000","last_became_peered":"2024-04-24T22:39:36.707676+0000","last_unstale":"2024-04-24T22:44:14.811730+0000","last_undegraded":"2024-04-24T22:44:14.811730+0000","last_fullsized":"2024-04-24T22:44:14.811730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-04-24T22:44:14.811622+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-04-24T22:44:14.811622+0000","last_clean_scrub_stamp":"2024-04-24T22:44:14.811622+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:18:23.524647+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'162","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:19.248710+0000","last_change":"2024-04-24T22:44:19.248775+0000","last_active":"2024-04-24T22:44:19.248710+0000","last_peered":"2024-04-24T22:44:19.248710+0000","last_clean":"2024-04-24T22:44:19.248710+0000","last_became_active":"2024-04-24T22:39:37.699820+0000","last_became_peered":"2024-04-24T22:39:37.699820+0000","last_unstale":"2024-04-24T22:44:19.248710+0000","last_undegraded":"2024-04-24T22:44:19.248710+0000","last_fullsized":"2024-04-24T22:44:19.248710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-04-24T22:44:19.248472+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-04-24T22:44:19.248472+0000","last_clean_scrub_stamp":"2024-04-24T22:44:19.248472+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:09.945323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:17.976057+0000","last_change":"2024-04-24T22:44:17.976130+0000","last_active":"2024-04-24T22:44:17.976057+0000","last_peered":"2024-04-24T22:44:17.976057+0000","last_clean":"2024-04-24T22:44:17.976057+0000","last_became_active":"2024-04-24T22:39:36.701056+0000","last_became_peered":"2024-04-24T22:39:36.701056+0000","last_unstale":"2024-04-24T22:44:17.976057+0000","last_undegraded":"2024-04-24T22:44:17.976057+0000","last_fullsized":"2024-04-24T22:44:17.976057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-04-24T22:44:17.975942+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-04-24T22:44:17.975942+0000","last_clean_scrub_stamp":"2024-04-24T22:44:17.975942+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:44.604486+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:15.774491+0000","last_change":"2024-04-24T22:44:15.774491+0000","last_active":"2024-04-24T22:44:15.774491+0000","last_peered":"2024-04-24T22:44:15.774491+0000","last_clean":"2024-04-24T22:44:15.774491+0000","last_became_active":"2024-04-24T22:39:36.712004+0000","last_became_peered":"2024-04-24T22:39:36.712004+0000","last_unstale":"2024-04-24T22:44:15.774491+0000","last_undegraded":"2024-04-24T22:44:15.774491+0000","last_fullsized":"2024-04-24T22:44:15.774491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-04-24T22:44:15.774042+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-04-24T22:44:15.774042+0000","last_clean_scrub_stamp":"2024-04-24T22:44:15.774042+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:47.324493+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'190","reported_seq":210,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:24.944787+0000","last_change":"2024-04-24T22:44:24.944908+0000","last_active":"2024-04-24T22:44:24.944787+0000","last_peered":"2024-04-24T22:44:24.944787+0000","last_clean":"2024-04-24T22:44:24.944787+0000","last_became_active":"2024-04-24T22:40:23.909573+0000","last_became_peered":"2024-04-24T22:40:23.909573+0000","last_unstale":"2024-04-24T22:44:24.944787+0000","last_undegraded":"2024-04-24T22:44:24.944787+0000","last_fullsized":"2024-04-24T22:44:24.944787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'190","last_scrub_stamp":"2024-04-24T22:44:24.944436+0000","last_deep_scrub":"23'190","last_deep_scrub_stamp":"2024-04-24T22:44:24.944436+0000","last_clean_scrub_stamp":"2024-04-24T22:44:24.944436+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:21.718675+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"23'4","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:15.837894+0000","last_change":"2024-04-24T22:44:15.837894+0000","last_active":"2024-04-24T22:44:15.837894+0000","last_peered":"2024-04-24T22:44:15.837894+0000","last_clean":"2024-04-24T22:44:15.837894+0000","last_became_active":"2024-04-24T22:39:36.698677+0000","last_became_peered":"2024-04-24T22:39:36.698677+0000","last_unstale":"2024-04-24T22:44:15.837894+0000","last_undegraded":"2024-04-24T22:44:15.837894+0000","last_fullsized":"2024-04-24T22:44:15.837894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-04-24T22:44:15.837730+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-04-24T22:44:15.837730+0000","last_clean_scrub_stamp":"2024-04-24T22:44:15.837730+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:24.203283+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'187","reported_seq":214,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:23.948004+0000","last_change":"2024-04-24T22:44:23.948004+0000","last_active":"2024-04-24T22:44:23.948004+0000","last_peered":"2024-04-24T22:44:23.948004+0000","last_clean":"2024-04-24T22:44:23.948004+0000","last_became_active":"2024-04-24T22:39:37.713923+0000","last_became_peered":"2024-04-24T22:39:37.713923+0000","last_unstale":"2024-04-24T22:44:23.948004+0000","last_undegraded":"2024-04-24T22:44:23.948004+0000","last_fullsized":"2024-04-24T22:44:23.948004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'187","last_scrub_stamp":"2024-04-24T22:44:23.947735+0000","last_deep_scrub":"23'187","last_deep_scrub_stamp":"2024-04-24T22:44:23.947735+0000","last_clean_scrub_stamp":"2024-04-24T22:44:23.947735+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:11.037377+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'8","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:18.990465+0000","last_change":"2024-04-24T22:44:18.990465+0000","last_active":"2024-04-24T22:44:18.990465+0000","last_peered":"2024-04-24T22:44:18.990465+0000","last_clean":"2024-04-24T22:44:18.990465+0000","last_became_active":"2024-04-24T22:39:36.705444+0000","last_became_peered":"2024-04-24T22:39:36.705444+0000","last_unstale":"2024-04-24T22:44:18.990465+0000","last_undegraded":"2024-04-24T22:44:18.990465+0000","last_fullsized":"2024-04-24T22:44:18.990465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-04-24T22:44:18.990301+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-04-24T22:44:18.990301+0000","last_clean_scrub_stamp":"2024-04-24T22:44:18.990301+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:14.387478+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'251","reported_seq":278,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:22.864445+0000","last_change":"2024-04-24T22:44:22.864445+0000","last_active":"2024-04-24T22:44:22.864445+0000","last_peered":"2024-04-24T22:44:22.864445+0000","last_clean":"2024-04-24T22:44:22.864445+0000","last_became_active":"2024-04-24T22:39:37.700369+0000","last_became_peered":"2024-04-24T22:39:37.700369+0000","last_unstale":"2024-04-24T22:44:22.864445+0000","last_undegraded":"2024-04-24T22:44:22.864445+0000","last_fullsized":"2024-04-24T22:44:22.864445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'251","last_scrub_stamp":"2024-04-24T22:44:22.864289+0000","last_deep_scrub":"23'251","last_deep_scrub_stamp":"2024-04-24T22:44:22.864289+0000","last_clean_scrub_stamp":"2024-04-24T22:44:22.864289+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:00.820620+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'6","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:16.836403+0000","last_change":"2024-04-24T22:44:16.836403+0000","last_active":"2024-04-24T22:44:16.836403+0000","last_peered":"2024-04-24T22:44:16.836403+0000","last_clean":"2024-04-24T22:44:16.836403+0000","last_became_active":"2024-04-24T22:39:36.699853+0000","last_became_peered":"2024-04-24T22:39:36.699853+0000","last_unstale":"2024-04-24T22:44:16.836403+0000","last_undegraded":"2024-04-24T22:44:16.836403+0000","last_fullsized":"2024-04-24T22:44:16.836403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-04-24T22:44:16.836278+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-04-24T22:44:16.836278+0000","last_clean_scrub_stamp":"2024-04-24T22:44:16.836278+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:00.133814+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'160","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:20.216246+0000","last_change":"2024-04-24T22:44:20.216246+0000","last_active":"2024-04-24T22:44:20.216246+0000","last_peered":"2024-04-24T22:44:20.216246+0000","last_clean":"2024-04-24T22:44:20.216246+0000","last_became_active":"2024-04-24T22:39:37.703121+0000","last_became_peered":"2024-04-24T22:39:37.703121+0000","last_unstale":"2024-04-24T22:44:20.216246+0000","last_undegraded":"2024-04-24T22:44:20.216246+0000","last_fullsized":"2024-04-24T22:44:20.216246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'160","last_scrub_stamp":"2024-04-24T22:44:20.215992+0000","last_deep_scrub":"23'160","last_deep_scrub_stamp":"2024-04-24T22:44:20.215992+0000","last_clean_scrub_stamp":"2024-04-24T22:44:20.215992+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:01.720362+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'14","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:14.872248+0000","last_change":"2024-04-24T22:44:14.872248+0000","last_active":"2024-04-24T22:44:14.872248+0000","last_peered":"2024-04-24T22:44:14.872248+0000","last_clean":"2024-04-24T22:44:14.872248+0000","last_became_active":"2024-04-24T22:39:36.716998+0000","last_became_peered":"2024-04-24T22:39:36.716998+0000","last_unstale":"2024-04-24T22:44:14.872248+0000","last_undegraded":"2024-04-24T22:44:14.872248+0000","last_fullsized":"2024-04-24T22:44:14.872248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-04-24T22:44:14.871909+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-04-24T22:44:14.871909+0000","last_clean_scrub_stamp":"2024-04-24T22:44:14.871909+0000","objects_scrubbed":4,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:53.404595+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'168","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:17.439482+0000","last_change":"2024-04-24T22:44:17.439482+0000","last_active":"2024-04-24T22:44:17.439482+0000","last_peered":"2024-04-24T22:44:17.439482+0000","last_clean":"2024-04-24T22:44:17.439482+0000","last_became_active":"2024-04-24T22:39:37.698941+0000","last_became_peered":"2024-04-24T22:39:37.698941+0000","last_unstale":"2024-04-24T22:44:17.439482+0000","last_undegraded":"2024-04-24T22:44:17.439482+0000","last_fullsized":"2024-04-24T22:44:17.439482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'168","last_scrub_stamp":"2024-04-24T22:44:17.439320+0000","last_deep_scrub":"23'168","last_deep_scrub_stamp":"2024-04-24T22:44:17.439320+0000","last_clean_scrub_stamp":"2024-04-24T22:44:17.439320+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:42.951645+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'173","reported_seq":202,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:20.026218+0000","last_change":"2024-04-24T22:44:20.026254+0000","last_active":"2024-04-24T22:44:20.026218+0000","last_peered":"2024-04-24T22:44:20.026218+0000","last_clean":"2024-04-24T22:44:20.026218+0000","last_became_active":"2024-04-24T22:39:36.706279+0000","last_became_peered":"2024-04-24T22:39:36.706279+0000","last_unstale":"2024-04-24T22:44:20.026218+0000","last_undegraded":"2024-04-24T22:44:20.026218+0000","last_fullsized":"2024-04-24T22:44:20.026218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'173","last_scrub_stamp":"2024-04-24T22:44:20.026125+0000","last_deep_scrub":"23'173","last_deep_scrub_stamp":"2024-04-24T22:44:20.026125+0000","last_clean_scrub_stamp":"2024-04-24T22:44:20.026125+0000","objects_scrubbed":6,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:28.391678+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":384079,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'176","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:33.408689+0000","last_change":"2024-04-24T22:44:33.408716+0000","last_active":"2024-04-24T22:44:33.408689+0000","last_peered":"2024-04-24T22:44:33.408689+0000","last_clean":"2024-04-24T22:44:33.408689+0000","last_became_active":"2024-04-24T22:39:37.711770+0000","last_became_peered":"2024-04-24T22:39:37.711770+0000","last_unstale":"2024-04-24T22:44:33.408689+0000","last_undegraded":"2024-04-24T22:44:33.408689+0000","last_fullsized":"2024-04-24T22:44:33.408689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'176","last_scrub_stamp":"2024-04-24T22:44:33.408570+0000","last_deep_scrub":"23'176","last_deep_scrub_stamp":"2024-04-24T22:44:33.408570+0000","last_clean_scrub_stamp":"2024-04-24T22:44:33.408570+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:02.543425+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'15","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:12.306242+0000","last_change":"2024-04-24T22:44:12.306242+0000","last_active":"2024-04-24T22:44:12.306242+0000","last_peered":"2024-04-24T22:44:12.306242+0000","last_clean":"2024-04-24T22:44:12.306242+0000","last_became_active":"2024-04-24T22:39:36.699673+0000","last_became_peered":"2024-04-24T22:39:36.699673+0000","last_unstale":"2024-04-24T22:44:12.306242+0000","last_undegraded":"2024-04-24T22:44:12.306242+0000","last_fullsized":"2024-04-24T22:44:12.306242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'15","last_scrub_stamp":"2024-04-24T22:44:12.306048+0000","last_deep_scrub":"23'15","last_deep_scrub_stamp":"2024-04-24T22:44:12.306048+0000","last_clean_scrub_stamp":"2024-04-24T22:44:12.306048+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:24.572459+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'177","reported_seq":204,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:14.258000+0000","last_change":"2024-04-24T22:44:14.258000+0000","last_active":"2024-04-24T22:44:14.258000+0000","last_peered":"2024-04-24T22:44:14.258000+0000","last_clean":"2024-04-24T22:44:14.258000+0000","last_became_active":"2024-04-24T22:39:37.691158+0000","last_became_peered":"2024-04-24T22:39:37.691158+0000","last_unstale":"2024-04-24T22:44:14.258000+0000","last_undegraded":"2024-04-24T22:44:14.258000+0000","last_fullsized":"2024-04-24T22:44:14.258000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'177","last_scrub_stamp":"2024-04-24T22:44:14.257836+0000","last_deep_scrub":"23'177","last_deep_scrub_stamp":"2024-04-24T22:44:14.257836+0000","last_clean_scrub_stamp":"2024-04-24T22:44:14.257836+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:52.790523+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'8","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:17.843061+0000","last_change":"2024-04-24T22:44:17.843061+0000","last_active":"2024-04-24T22:44:17.843061+0000","last_peered":"2024-04-24T22:44:17.843061+0000","last_clean":"2024-04-24T22:44:17.843061+0000","last_became_active":"2024-04-24T22:39:36.706141+0000","last_became_peered":"2024-04-24T22:39:36.706141+0000","last_unstale":"2024-04-24T22:44:17.843061+0000","last_undegraded":"2024-04-24T22:44:17.843061+0000","last_fullsized":"2024-04-24T22:44:17.843061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-04-24T22:44:17.842920+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-04-24T22:44:17.842920+0000","last_clean_scrub_stamp":"2024-04-24T22:44:17.842920+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:08.539771+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'201","reported_seq":228,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:25.483396+0000","last_change":"2024-04-24T22:44:25.483421+0000","last_active":"2024-04-24T22:44:25.483396+0000","last_peered":"2024-04-24T22:44:25.483396+0000","last_clean":"2024-04-24T22:44:25.483396+0000","last_became_active":"2024-04-24T22:39:37.696275+0000","last_became_peered":"2024-04-24T22:39:37.696275+0000","last_unstale":"2024-04-24T22:44:25.483396+0000","last_undegraded":"2024-04-24T22:44:25.483396+0000","last_fullsized":"2024-04-24T22:44:25.483396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'201","last_scrub_stamp":"2024-04-24T22:44:25.483300+0000","last_deep_scrub":"23'201","last_deep_scrub_stamp":"2024-04-24T22:44:25.483300+0000","last_clean_scrub_stamp":"2024-04-24T22:44:25.483300+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:16.757139+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'8","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:16.947754+0000","last_change":"2024-04-24T22:44:16.947754+0000","last_active":"2024-04-24T22:44:16.947754+0000","last_peered":"2024-04-24T22:44:16.947754+0000","last_clean":"2024-04-24T22:44:16.947754+0000","last_became_active":"2024-04-24T22:39:36.701020+0000","last_became_peered":"2024-04-24T22:39:36.701020+0000","last_unstale":"2024-04-24T22:44:16.947754+0000","last_undegraded":"2024-04-24T22:44:16.947754+0000","last_fullsized":"2024-04-24T22:44:16.947754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-04-24T22:44:16.947635+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-04-24T22:44:16.947635+0000","last_clean_scrub_stamp":"2024-04-24T22:44:16.947635+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:49:03.443401+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'197","reported_seq":224,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:22.928358+0000","last_change":"2024-04-24T22:44:22.928386+0000","last_active":"2024-04-24T22:44:22.928358+0000","last_peered":"2024-04-24T22:44:22.928358+0000","last_clean":"2024-04-24T22:44:22.928358+0000","last_became_active":"2024-04-24T22:39:37.697235+0000","last_became_peered":"2024-04-24T22:39:37.697235+0000","last_unstale":"2024-04-24T22:44:22.928358+0000","last_undegraded":"2024-04-24T22:44:22.928358+0000","last_fullsized":"2024-04-24T22:44:22.928358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'197","last_scrub_stamp":"2024-04-24T22:44:22.928244+0000","last_deep_scrub":"23'197","last_deep_scrub_stamp":"2024-04-24T22:44:22.928244+0000","last_clean_scrub_stamp":"2024-04-24T22:44:22.928244+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:01.826835+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'26","reported_seq":55,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:15.908224+0000","last_change":"2024-04-24T22:44:15.908224+0000","last_active":"2024-04-24T22:44:15.908224+0000","last_peered":"2024-04-24T22:44:15.908224+0000","last_clean":"2024-04-24T22:44:15.908224+0000","last_became_active":"2024-04-24T22:39:36.724302+0000","last_became_peered":"2024-04-24T22:39:36.724302+0000","last_unstale":"2024-04-24T22:44:15.908224+0000","last_undegraded":"2024-04-24T22:44:15.908224+0000","last_fullsized":"2024-04-24T22:44:15.908224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'26","last_scrub_stamp":"2024-04-24T22:44:15.908072+0000","last_deep_scrub":"23'26","last_deep_scrub_stamp":"2024-04-24T22:44:15.908072+0000","last_clean_scrub_stamp":"2024-04-24T22:44:15.908072+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:06.774780+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'215","reported_seq":242,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:26.482797+0000","last_change":"2024-04-24T22:44:26.482824+0000","last_active":"2024-04-24T22:44:26.482797+0000","last_peered":"2024-04-24T22:44:26.482797+0000","last_clean":"2024-04-24T22:44:26.482797+0000","last_became_active":"2024-04-24T22:39:37.702644+0000","last_became_peered":"2024-04-24T22:39:37.702644+0000","last_unstale":"2024-04-24T22:44:26.482797+0000","last_undegraded":"2024-04-24T22:44:26.482797+0000","last_fullsized":"2024-04-24T22:44:26.482797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'215","last_scrub_stamp":"2024-04-24T22:44:26.482699+0000","last_deep_scrub":"23'215","last_deep_scrub_stamp":"2024-04-24T22:44:26.482699+0000","last_clean_scrub_stamp":"2024-04-24T22:44:26.482699+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:50:37.992841+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'4","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:15.378624+0000","last_change":"2024-04-24T22:44:15.378657+0000","last_active":"2024-04-24T22:44:15.378624+0000","last_peered":"2024-04-24T22:44:15.378624+0000","last_clean":"2024-04-24T22:44:15.378624+0000","last_became_active":"2024-04-24T22:39:36.715732+0000","last_became_peered":"2024-04-24T22:39:36.715732+0000","last_unstale":"2024-04-24T22:44:15.378624+0000","last_undegraded":"2024-04-24T22:44:15.378624+0000","last_fullsized":"2024-04-24T22:44:15.378624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-04-24T22:44:15.378521+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-04-24T22:44:15.378521+0000","last_clean_scrub_stamp":"2024-04-24T22:44:15.378521+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:45.343285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'175","reported_seq":202,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:17.477797+0000","last_change":"2024-04-24T22:44:17.477856+0000","last_active":"2024-04-24T22:44:17.477797+0000","last_peered":"2024-04-24T22:44:17.477797+0000","last_clean":"2024-04-24T22:44:17.477797+0000","last_became_active":"2024-04-24T22:39:37.702293+0000","last_became_peered":"2024-04-24T22:39:37.702293+0000","last_unstale":"2024-04-24T22:44:17.477797+0000","last_undegraded":"2024-04-24T22:44:17.477797+0000","last_fullsized":"2024-04-24T22:44:17.477797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'175","last_scrub_stamp":"2024-04-24T22:44:17.477667+0000","last_deep_scrub":"23'175","last_deep_scrub_stamp":"2024-04-24T22:44:17.477667+0000","last_clean_scrub_stamp":"2024-04-24T22:44:17.477667+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:11:25.843220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:14.867184+0000","last_change":"2024-04-24T22:44:14.867233+0000","last_active":"2024-04-24T22:44:14.867184+0000","last_peered":"2024-04-24T22:44:14.867184+0000","last_clean":"2024-04-24T22:44:14.867184+0000","last_became_active":"2024-04-24T22:39:36.713114+0000","last_became_peered":"2024-04-24T22:39:36.713114+0000","last_unstale":"2024-04-24T22:44:14.867184+0000","last_undegraded":"2024-04-24T22:44:14.867184+0000","last_fullsized":"2024-04-24T22:44:14.867184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-04-24T22:44:14.867080+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-04-24T22:44:14.867080+0000","last_clean_scrub_stamp":"2024-04-24T22:44:14.867080+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:23.622663+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'162","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:28.476094+0000","last_change":"2024-04-24T22:44:28.476138+0000","last_active":"2024-04-24T22:44:28.476094+0000","last_peered":"2024-04-24T22:44:28.476094+0000","last_clean":"2024-04-24T22:44:28.476094+0000","last_became_active":"2024-04-24T22:39:37.703118+0000","last_became_peered":"2024-04-24T22:39:37.703118+0000","last_unstale":"2024-04-24T22:44:28.476094+0000","last_undegraded":"2024-04-24T22:44:28.476094+0000","last_fullsized":"2024-04-24T22:44:28.476094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-04-24T22:44:28.475950+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-04-24T22:44:28.475950+0000","last_clean_scrub_stamp":"2024-04-24T22:44:28.475950+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:33.371961+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'4","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:12.829705+0000","last_change":"2024-04-24T22:44:12.829772+0000","last_active":"2024-04-24T22:44:12.829705+0000","last_peered":"2024-04-24T22:44:12.829705+0000","last_clean":"2024-04-24T22:44:12.829705+0000","last_became_active":"2024-04-24T22:39:36.713563+0000","last_became_peered":"2024-04-24T22:39:36.713563+0000","last_unstale":"2024-04-24T22:44:12.829705+0000","last_undegraded":"2024-04-24T22:44:12.829705+0000","last_fullsized":"2024-04-24T22:44:12.829705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-04-24T22:44:12.829431+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-04-24T22:44:12.829431+0000","last_clean_scrub_stamp":"2024-04-24T22:44:12.829431+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:25.095842+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'165","reported_seq":192,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:28.898124+0000","last_change":"2024-04-24T22:44:28.898149+0000","last_active":"2024-04-24T22:44:28.898124+0000","last_peered":"2024-04-24T22:44:28.898124+0000","last_clean":"2024-04-24T22:44:28.898124+0000","last_became_active":"2024-04-24T22:39:37.715575+0000","last_became_peered":"2024-04-24T22:39:37.715575+0000","last_unstale":"2024-04-24T22:44:28.898124+0000","last_undegraded":"2024-04-24T22:44:28.898124+0000","last_fullsized":"2024-04-24T22:44:28.898124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'165","last_scrub_stamp":"2024-04-24T22:44:28.897996+0000","last_deep_scrub":"23'165","last_deep_scrub_stamp":"2024-04-24T22:44:28.897996+0000","last_clean_scrub_stamp":"2024-04-24T22:44:28.897996+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:36.019423+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'8","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:13.874883+0000","last_change":"2024-04-24T22:44:13.874930+0000","last_active":"2024-04-24T22:44:13.874883+0000","last_peered":"2024-04-24T22:44:13.874883+0000","last_clean":"2024-04-24T22:44:13.874883+0000","last_became_active":"2024-04-24T22:39:36.710087+0000","last_became_peered":"2024-04-24T22:39:36.710087+0000","last_unstale":"2024-04-24T22:44:13.874883+0000","last_undegraded":"2024-04-24T22:44:13.874883+0000","last_fullsized":"2024-04-24T22:44:13.874883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-04-24T22:44:13.874775+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-04-24T22:44:13.874775+0000","last_clean_scrub_stamp":"2024-04-24T22:44:13.874775+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:04.186004+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'175","reported_seq":202,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:31.457506+0000","last_change":"2024-04-24T22:44:31.457533+0000","last_active":"2024-04-24T22:44:31.457506+0000","last_peered":"2024-04-24T22:44:31.457506+0000","last_clean":"2024-04-24T22:44:31.457506+0000","last_became_active":"2024-04-24T22:39:37.706328+0000","last_became_peered":"2024-04-24T22:39:37.706328+0000","last_unstale":"2024-04-24T22:44:31.457506+0000","last_undegraded":"2024-04-24T22:44:31.457506+0000","last_fullsized":"2024-04-24T22:44:31.457506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'175","last_scrub_stamp":"2024-04-24T22:44:31.457403+0000","last_deep_scrub":"23'175","last_deep_scrub_stamp":"2024-04-24T22:44:31.457403+0000","last_clean_scrub_stamp":"2024-04-24T22:44:31.457403+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:55.393451+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'119","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:18.445813+0000","last_change":"2024-04-24T22:44:18.445847+0000","last_active":"2024-04-24T22:44:18.445813+0000","last_peered":"2024-04-24T22:44:18.445813+0000","last_clean":"2024-04-24T22:44:18.445813+0000","last_became_active":"2024-04-24T22:39:32.656950+0000","last_became_peered":"2024-04-24T22:39:32.656950+0000","last_unstale":"2024-04-24T22:44:18.445813+0000","last_undegraded":"2024-04-24T22:44:18.445813+0000","last_fullsized":"2024-04-24T22:44:18.445813+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'119","last_scrub_stamp":"2024-04-24T22:44:18.445723+0000","last_deep_scrub":"23'119","last_deep_scrub_stamp":"2024-04-24T22:44:18.445723+0000","last_clean_scrub_stamp":"2024-04-24T22:44:18.445723+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:27.210982+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'11","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:18.888103+0000","last_change":"2024-04-24T22:44:18.888141+0000","last_active":"2024-04-24T22:44:18.888103+0000","last_peered":"2024-04-24T22:44:18.888103+0000","last_clean":"2024-04-24T22:44:18.888103+0000","last_became_active":"2024-04-24T22:39:36.710008+0000","last_became_peered":"2024-04-24T22:39:36.710008+0000","last_unstale":"2024-04-24T22:44:18.888103+0000","last_undegraded":"2024-04-24T22:44:18.888103+0000","last_fullsized":"2024-04-24T22:44:18.888103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11","last_scrub_stamp":"2024-04-24T22:44:18.888003+0000","last_deep_scrub":"23'11","last_deep_scrub_stamp":"2024-04-24T22:44:18.888003+0000","last_clean_scrub_stamp":"2024-04-24T22:44:18.888003+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:00.392374+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'183","reported_seq":210,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:21.694059+0000","last_change":"2024-04-24T22:44:21.694119+0000","last_active":"2024-04-24T22:44:21.694059+0000","last_peered":"2024-04-24T22:44:21.694059+0000","last_clean":"2024-04-24T22:44:21.694059+0000","last_became_active":"2024-04-24T22:39:37.713687+0000","last_became_peered":"2024-04-24T22:39:37.713687+0000","last_unstale":"2024-04-24T22:44:21.694059+0000","last_undegraded":"2024-04-24T22:44:21.694059+0000","last_fullsized":"2024-04-24T22:44:21.694059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'183","last_scrub_stamp":"2024-04-24T22:44:21.693867+0000","last_deep_scrub":"23'183","last_deep_scrub_stamp":"2024-04-24T22:44:21.693867+0000","last_clean_scrub_stamp":"2024-04-24T22:44:21.693867+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:01.631443+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:22.500230+0000","last_change":"2024-04-24T22:44:22.500258+0000","last_active":"2024-04-24T22:44:22.500230+0000","last_peered":"2024-04-24T22:44:22.500230+0000","last_clean":"2024-04-24T22:44:22.500230+0000","last_became_active":"2024-04-24T22:39:36.715062+0000","last_became_peered":"2024-04-24T22:39:36.715062+0000","last_unstale":"2024-04-24T22:44:22.500230+0000","last_undegraded":"2024-04-24T22:44:22.500230+0000","last_fullsized":"2024-04-24T22:44:22.500230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-04-24T22:44:22.500146+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-04-24T22:44:22.500146+0000","last_clean_scrub_stamp":"2024-04-24T22:44:22.500146+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:20:02.437721+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'165","reported_seq":192,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:15.265829+0000","last_change":"2024-04-24T22:44:15.265887+0000","last_active":"2024-04-24T22:44:15.265829+0000","last_peered":"2024-04-24T22:44:15.265829+0000","last_clean":"2024-04-24T22:44:15.265829+0000","last_became_active":"2024-04-24T22:39:37.693834+0000","last_became_peered":"2024-04-24T22:39:37.693834+0000","last_unstale":"2024-04-24T22:44:15.265829+0000","last_undegraded":"2024-04-24T22:44:15.265829+0000","last_fullsized":"2024-04-24T22:44:15.265829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'165","last_scrub_stamp":"2024-04-24T22:44:15.265510+0000","last_deep_scrub":"23'165","last_deep_scrub_stamp":"2024-04-24T22:44:15.265510+0000","last_clean_scrub_stamp":"2024-04-24T22:44:15.265510+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:59.568679+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'14","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:12.893018+0000","last_change":"2024-04-24T22:44:12.893090+0000","last_active":"2024-04-24T22:44:12.893018+0000","last_peered":"2024-04-24T22:44:12.893018+0000","last_clean":"2024-04-24T22:44:12.893018+0000","last_became_active":"2024-04-24T22:39:36.707232+0000","last_became_peered":"2024-04-24T22:39:36.707232+0000","last_unstale":"2024-04-24T22:44:12.893018+0000","last_undegraded":"2024-04-24T22:44:12.893018+0000","last_fullsized":"2024-04-24T22:44:12.893018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-04-24T22:44:12.892835+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-04-24T22:44:12.892835+0000","last_clean_scrub_stamp":"2024-04-24T22:44:12.892835+0000","objects_scrubbed":4,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:10:45.813715+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'165","reported_seq":192,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:20.942056+0000","last_change":"2024-04-24T22:44:20.942180+0000","last_active":"2024-04-24T22:44:20.942056+0000","last_peered":"2024-04-24T22:44:20.942056+0000","last_clean":"2024-04-24T22:44:20.942056+0000","last_became_active":"2024-04-24T22:39:37.703006+0000","last_became_peered":"2024-04-24T22:39:37.703006+0000","last_unstale":"2024-04-24T22:44:20.942056+0000","last_undegraded":"2024-04-24T22:44:20.942056+0000","last_fullsized":"2024-04-24T22:44:20.942056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'165","last_scrub_stamp":"2024-04-24T22:44:20.941789+0000","last_deep_scrub":"23'165","last_deep_scrub_stamp":"2024-04-24T22:44:20.941789+0000","last_clean_scrub_stamp":"2024-04-24T22:44:20.941789+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:02.446159+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'8","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:19.879341+0000","last_change":"2024-04-24T22:44:19.879373+0000","last_active":"2024-04-24T22:44:19.879341+0000","last_peered":"2024-04-24T22:44:19.879341+0000","last_clean":"2024-04-24T22:44:19.879341+0000","last_became_active":"2024-04-24T22:39:36.713148+0000","last_became_peered":"2024-04-24T22:39:36.713148+0000","last_unstale":"2024-04-24T22:44:19.879341+0000","last_undegraded":"2024-04-24T22:44:19.879341+0000","last_fullsized":"2024-04-24T22:44:19.879341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-04-24T22:44:19.879248+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-04-24T22:44:19.879248+0000","last_clean_scrub_stamp":"2024-04-24T22:44:19.879248+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:59.749171+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":987,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'177","reported_seq":204,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:20.476328+0000","last_change":"2024-04-24T22:44:20.476355+0000","last_active":"2024-04-24T22:44:20.476328+0000","last_peered":"2024-04-24T22:44:20.476328+0000","last_clean":"2024-04-24T22:44:20.476328+0000","last_became_active":"2024-04-24T22:39:37.705457+0000","last_became_peered":"2024-04-24T22:39:37.705457+0000","last_unstale":"2024-04-24T22:44:20.476328+0000","last_undegraded":"2024-04-24T22:44:20.476328+0000","last_fullsized":"2024-04-24T22:44:20.476328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'177","last_scrub_stamp":"2024-04-24T22:44:20.476239+0000","last_deep_scrub":"23'177","last_deep_scrub_stamp":"2024-04-24T22:44:20.476239+0000","last_clean_scrub_stamp":"2024-04-24T22:44:20.476239+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:32.742650+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:17.908583+0000","last_change":"2024-04-24T22:44:17.908659+0000","last_active":"2024-04-24T22:44:17.908583+0000","last_peered":"2024-04-24T22:44:17.908583+0000","last_clean":"2024-04-24T22:44:17.908583+0000","last_became_active":"2024-04-24T22:39:36.722800+0000","last_became_peered":"2024-04-24T22:39:36.722800+0000","last_unstale":"2024-04-24T22:44:17.908583+0000","last_undegraded":"2024-04-24T22:44:17.908583+0000","last_fullsized":"2024-04-24T22:44:17.908583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-04-24T22:44:17.908348+0000","last_deep_scrub":"21'1","last_deep_scrub_stamp":"2024-04-24T22:44:17.908348+0000","last_clean_scrub_stamp":"2024-04-24T22:44:17.908348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:58.606847+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'160","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:27.884513+0000","last_change":"2024-04-24T22:44:27.884537+0000","last_active":"2024-04-24T22:44:27.884513+0000","last_peered":"2024-04-24T22:44:27.884513+0000","last_clean":"2024-04-24T22:44:27.884513+0000","last_became_active":"2024-04-24T22:39:37.713542+0000","last_became_peered":"2024-04-24T22:39:37.713542+0000","last_unstale":"2024-04-24T22:44:27.884513+0000","last_undegraded":"2024-04-24T22:44:27.884513+0000","last_fullsized":"2024-04-24T22:44:27.884513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'160","last_scrub_stamp":"2024-04-24T22:44:27.884420+0000","last_deep_scrub":"23'160","last_deep_scrub_stamp":"2024-04-24T22:44:27.884420+0000","last_clean_scrub_stamp":"2024-04-24T22:44:27.884420+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:26.640273+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:24.479011+0000","last_change":"2024-04-24T22:44:24.479060+0000","last_active":"2024-04-24T22:44:24.479011+0000","last_peered":"2024-04-24T22:44:24.479011+0000","last_clean":"2024-04-24T22:44:24.479011+0000","last_became_active":"2024-04-24T22:39:36.715893+0000","last_became_peered":"2024-04-24T22:39:36.715893+0000","last_unstale":"2024-04-24T22:44:24.479011+0000","last_undegraded":"2024-04-24T22:44:24.479011+0000","last_fullsized":"2024-04-24T22:44:24.479011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-04-24T22:44:24.478871+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-04-24T22:44:24.478871+0000","last_clean_scrub_stamp":"2024-04-24T22:44:24.478871+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:02.435202+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'162","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:21.902839+0000","last_change":"2024-04-24T22:44:21.902964+0000","last_active":"2024-04-24T22:44:21.902839+0000","last_peered":"2024-04-24T22:44:21.902839+0000","last_clean":"2024-04-24T22:44:21.902839+0000","last_became_active":"2024-04-24T22:39:37.707746+0000","last_became_peered":"2024-04-24T22:39:37.707746+0000","last_unstale":"2024-04-24T22:44:21.902839+0000","last_undegraded":"2024-04-24T22:44:21.902839+0000","last_fullsized":"2024-04-24T22:44:21.902839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-04-24T22:44:21.902589+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-04-24T22:44:21.902589+0000","last_clean_scrub_stamp":"2024-04-24T22:44:21.902589+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:01.772512+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'31","reported_seq":60,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:15.899015+0000","last_change":"2024-04-24T22:44:15.899053+0000","last_active":"2024-04-24T22:44:15.899015+0000","last_peered":"2024-04-24T22:44:15.899015+0000","last_clean":"2024-04-24T22:44:15.899015+0000","last_became_active":"2024-04-24T22:39:36.718356+0000","last_became_peered":"2024-04-24T22:39:36.718356+0000","last_unstale":"2024-04-24T22:44:15.899015+0000","last_undegraded":"2024-04-24T22:44:15.899015+0000","last_fullsized":"2024-04-24T22:44:15.899015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'31","last_scrub_stamp":"2024-04-24T22:44:15.898888+0000","last_deep_scrub":"23'31","last_deep_scrub_stamp":"2024-04-24T22:44:15.898888+0000","last_clean_scrub_stamp":"2024-04-24T22:44:15.898888+0000","objects_scrubbed":4,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:00.202056+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'166","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:16.785103+0000","last_change":"2024-04-24T22:44:16.785160+0000","last_active":"2024-04-24T22:44:16.785103+0000","last_peered":"2024-04-24T22:44:16.785103+0000","last_clean":"2024-04-24T22:44:16.785103+0000","last_became_active":"2024-04-24T22:39:37.703603+0000","last_became_peered":"2024-04-24T22:39:37.703603+0000","last_unstale":"2024-04-24T22:44:16.785103+0000","last_undegraded":"2024-04-24T22:44:16.785103+0000","last_fullsized":"2024-04-24T22:44:16.785103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'166","last_scrub_stamp":"2024-04-24T22:44:16.784906+0000","last_deep_scrub":"23'166","last_deep_scrub_stamp":"2024-04-24T22:44:16.784906+0000","last_clean_scrub_stamp":"2024-04-24T22:44:16.784906+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:57.483405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'12","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:15.996801+0000","last_change":"2024-04-24T22:44:15.996855+0000","last_active":"2024-04-24T22:44:15.996801+0000","last_peered":"2024-04-24T22:44:15.996801+0000","last_clean":"2024-04-24T22:44:15.996801+0000","last_became_active":"2024-04-24T22:39:36.700987+0000","last_became_peered":"2024-04-24T22:39:36.700987+0000","last_unstale":"2024-04-24T22:44:15.996801+0000","last_undegraded":"2024-04-24T22:44:15.996801+0000","last_fullsized":"2024-04-24T22:44:15.996801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-04-24T22:44:15.996685+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-04-24T22:44:15.996685+0000","last_clean_scrub_stamp":"2024-04-24T22:44:15.996685+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:36.232988+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'183","reported_seq":210,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:18.221781+0000","last_change":"2024-04-24T22:44:18.221907+0000","last_active":"2024-04-24T22:44:18.221781+0000","last_peered":"2024-04-24T22:44:18.221781+0000","last_clean":"2024-04-24T22:44:18.221781+0000","last_became_active":"2024-04-24T22:39:37.699894+0000","last_became_peered":"2024-04-24T22:39:37.699894+0000","last_unstale":"2024-04-24T22:44:18.221781+0000","last_undegraded":"2024-04-24T22:44:18.221781+0000","last_fullsized":"2024-04-24T22:44:18.221781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'183","last_scrub_stamp":"2024-04-24T22:44:18.221517+0000","last_deep_scrub":"23'183","last_deep_scrub_stamp":"2024-04-24T22:44:18.221517+0000","last_clean_scrub_stamp":"2024-04-24T22:44:18.221517+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:26.986776+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:23.499355+0000","last_change":"2024-04-24T22:44:23.499388+0000","last_active":"2024-04-24T22:44:23.499355+0000","last_peered":"2024-04-24T22:44:23.499355+0000","last_clean":"2024-04-24T22:44:23.499355+0000","last_became_active":"2024-04-24T22:39:36.715864+0000","last_became_peered":"2024-04-24T22:39:36.715864+0000","last_unstale":"2024-04-24T22:44:23.499355+0000","last_undegraded":"2024-04-24T22:44:23.499355+0000","last_fullsized":"2024-04-24T22:44:23.499355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-04-24T22:44:23.499266+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-04-24T22:44:23.499266+0000","last_clean_scrub_stamp":"2024-04-24T22:44:23.499266+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:27.459631+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'162","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:18.475361+0000","last_change":"2024-04-24T22:44:18.475386+0000","last_active":"2024-04-24T22:44:18.475361+0000","last_peered":"2024-04-24T22:44:18.475361+0000","last_clean":"2024-04-24T22:44:18.475361+0000","last_became_active":"2024-04-24T22:39:37.703853+0000","last_became_peered":"2024-04-24T22:39:37.703853+0000","last_unstale":"2024-04-24T22:44:18.475361+0000","last_undegraded":"2024-04-24T22:44:18.475361+0000","last_fullsized":"2024-04-24T22:44:18.475361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-04-24T22:44:18.475256+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-04-24T22:44:18.475256+0000","last_clean_scrub_stamp":"2024-04-24T22:44:18.475256+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:32:52.254489+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'11","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:16.900975+0000","last_change":"2024-04-24T22:44:16.901049+0000","last_active":"2024-04-24T22:44:16.900975+0000","last_peered":"2024-04-24T22:44:16.900975+0000","last_clean":"2024-04-24T22:44:16.900975+0000","last_became_active":"2024-04-24T22:39:36.721333+0000","last_became_peered":"2024-04-24T22:39:36.721333+0000","last_unstale":"2024-04-24T22:44:16.900975+0000","last_undegraded":"2024-04-24T22:44:16.900975+0000","last_fullsized":"2024-04-24T22:44:16.900975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11","last_scrub_stamp":"2024-04-24T22:44:16.900772+0000","last_deep_scrub":"23'11","last_deep_scrub_stamp":"2024-04-24T22:44:16.900772+0000","last_clean_scrub_stamp":"2024-04-24T22:44:16.900772+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:48.451224+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'173","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:22.675502+0000","last_change":"2024-04-24T22:44:22.675587+0000","last_active":"2024-04-24T22:44:22.675502+0000","last_peered":"2024-04-24T22:44:22.675502+0000","last_clean":"2024-04-24T22:44:22.675502+0000","last_became_active":"2024-04-24T22:39:37.713896+0000","last_became_peered":"2024-04-24T22:39:37.713896+0000","last_unstale":"2024-04-24T22:44:22.675502+0000","last_undegraded":"2024-04-24T22:44:22.675502+0000","last_fullsized":"2024-04-24T22:44:22.675502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'173","last_scrub_stamp":"2024-04-24T22:44:22.675309+0000","last_deep_scrub":"23'173","last_deep_scrub_stamp":"2024-04-24T22:44:22.675309+0000","last_clean_scrub_stamp":"2024-04-24T22:44:22.675309+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:26.635674+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:13.837292+0000","last_change":"2024-04-24T22:44:13.837432+0000","last_active":"2024-04-24T22:44:13.837292+0000","last_peered":"2024-04-24T22:44:13.837292+0000","last_clean":"2024-04-24T22:44:13.837292+0000","last_became_active":"2024-04-24T22:39:36.715506+0000","last_became_peered":"2024-04-24T22:39:36.715506+0000","last_unstale":"2024-04-24T22:44:13.837292+0000","last_undegraded":"2024-04-24T22:44:13.837292+0000","last_fullsized":"2024-04-24T22:44:13.837292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-04-24T22:44:13.836965+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-04-24T22:44:13.836965+0000","last_clean_scrub_stamp":"2024-04-24T22:44:13.836965+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:01.881690+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'164","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:24.645338+0000","last_change":"2024-04-24T22:44:24.645399+0000","last_active":"2024-04-24T22:44:24.645338+0000","last_peered":"2024-04-24T22:44:24.645338+0000","last_clean":"2024-04-24T22:44:24.645338+0000","last_became_active":"2024-04-24T22:39:37.715372+0000","last_became_peered":"2024-04-24T22:39:37.715372+0000","last_unstale":"2024-04-24T22:44:24.645338+0000","last_undegraded":"2024-04-24T22:44:24.645338+0000","last_fullsized":"2024-04-24T22:44:24.645338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'164","last_scrub_stamp":"2024-04-24T22:44:24.645147+0000","last_deep_scrub":"23'164","last_deep_scrub_stamp":"2024-04-24T22:44:24.645147+0000","last_clean_scrub_stamp":"2024-04-24T22:44:24.645147+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:15.918986+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:19.444144+0000","last_change":"2024-04-24T22:44:19.444181+0000","last_active":"2024-04-24T22:44:19.444144+0000","last_peered":"2024-04-24T22:44:19.444144+0000","last_clean":"2024-04-24T22:44:19.444144+0000","last_became_active":"2024-04-24T22:39:36.697897+0000","last_became_peered":"2024-04-24T22:39:36.697897+0000","last_unstale":"2024-04-24T22:44:19.444144+0000","last_undegraded":"2024-04-24T22:44:19.444144+0000","last_fullsized":"2024-04-24T22:44:19.444144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-04-24T22:44:19.444045+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-04-24T22:44:19.444045+0000","last_clean_scrub_stamp":"2024-04-24T22:44:19.444045+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:10.140896+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'154","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:20.986476+0000","last_change":"2024-04-24T22:44:20.986514+0000","last_active":"2024-04-24T22:44:20.986476+0000","last_peered":"2024-04-24T22:44:20.986476+0000","last_clean":"2024-04-24T22:44:20.986476+0000","last_became_active":"2024-04-24T22:39:37.695846+0000","last_became_peered":"2024-04-24T22:39:37.695846+0000","last_unstale":"2024-04-24T22:44:20.986476+0000","last_undegraded":"2024-04-24T22:44:20.986476+0000","last_fullsized":"2024-04-24T22:44:20.986476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'154","last_scrub_stamp":"2024-04-24T22:44:20.986377+0000","last_deep_scrub":"23'154","last_deep_scrub_stamp":"2024-04-24T22:44:20.986377+0000","last_clean_scrub_stamp":"2024-04-24T22:44:20.986377+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:04.283912+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'205","reported_seq":231,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:20.913215+0000","last_change":"2024-04-24T22:44:20.913242+0000","last_active":"2024-04-24T22:44:20.913215+0000","last_peered":"2024-04-24T22:44:20.913215+0000","last_clean":"2024-04-24T22:44:20.913215+0000","last_became_active":"2024-04-24T22:39:37.695633+0000","last_became_peered":"2024-04-24T22:39:37.695633+0000","last_unstale":"2024-04-24T22:44:20.913215+0000","last_undegraded":"2024-04-24T22:44:20.913215+0000","last_fullsized":"2024-04-24T22:44:20.913215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'205","last_scrub_stamp":"2024-04-24T22:44:20.913111+0000","last_deep_scrub":"23'205","last_deep_scrub_stamp":"2024-04-24T22:44:20.913111+0000","last_clean_scrub_stamp":"2024-04-24T22:44:20.913111+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:59.247170+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'8","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:21.530956+0000","last_change":"2024-04-24T22:44:21.530989+0000","last_active":"2024-04-24T22:44:21.530956+0000","last_peered":"2024-04-24T22:44:21.530956+0000","last_clean":"2024-04-24T22:44:21.530956+0000","last_became_active":"2024-04-24T22:39:36.698871+0000","last_became_peered":"2024-04-24T22:39:36.698871+0000","last_unstale":"2024-04-24T22:44:21.530956+0000","last_undegraded":"2024-04-24T22:44:21.530956+0000","last_fullsized":"2024-04-24T22:44:21.530956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-04-24T22:44:21.530865+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-04-24T22:44:21.530865+0000","last_clean_scrub_stamp":"2024-04-24T22:44:21.530865+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:35.956116+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'162","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:21.876442+0000","last_change":"2024-04-24T22:44:21.876469+0000","last_active":"2024-04-24T22:44:21.876442+0000","last_peered":"2024-04-24T22:44:21.876442+0000","last_clean":"2024-04-24T22:44:21.876442+0000","last_became_active":"2024-04-24T22:39:37.700688+0000","last_became_peered":"2024-04-24T22:39:37.700688+0000","last_unstale":"2024-04-24T22:44:21.876442+0000","last_undegraded":"2024-04-24T22:44:21.876442+0000","last_fullsized":"2024-04-24T22:44:21.876442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-04-24T22:44:21.876332+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-04-24T22:44:21.876332+0000","last_clean_scrub_stamp":"2024-04-24T22:44:21.876332+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:08.965975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'21","reported_seq":49,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:20.482224+0000","last_change":"2024-04-24T22:44:20.482266+0000","last_active":"2024-04-24T22:44:20.482224+0000","last_peered":"2024-04-24T22:44:20.482224+0000","last_clean":"2024-04-24T22:44:20.482224+0000","last_became_active":"2024-04-24T22:39:36.698173+0000","last_became_peered":"2024-04-24T22:39:36.698173+0000","last_unstale":"2024-04-24T22:44:20.482224+0000","last_undegraded":"2024-04-24T22:44:20.482224+0000","last_fullsized":"2024-04-24T22:44:20.482224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'21","last_scrub_stamp":"2024-04-24T22:44:20.482125+0000","last_deep_scrub":"23'21","last_deep_scrub_stamp":"2024-04-24T22:44:20.482125+0000","last_clean_scrub_stamp":"2024-04-24T22:44:20.482125+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:14:41.143760+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'160","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:21.167800+0000","last_change":"2024-04-24T22:44:21.167859+0000","last_active":"2024-04-24T22:44:21.167800+0000","last_peered":"2024-04-24T22:44:21.167800+0000","last_clean":"2024-04-24T22:44:21.167800+0000","last_became_active":"2024-04-24T22:39:37.705996+0000","last_became_peered":"2024-04-24T22:39:37.705996+0000","last_unstale":"2024-04-24T22:44:21.167800+0000","last_undegraded":"2024-04-24T22:44:21.167800+0000","last_fullsized":"2024-04-24T22:44:21.167800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'160","last_scrub_stamp":"2024-04-24T22:44:21.167539+0000","last_deep_scrub":"23'160","last_deep_scrub_stamp":"2024-04-24T22:44:21.167539+0000","last_clean_scrub_stamp":"2024-04-24T22:44:21.167539+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:35.592790+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:13.839259+0000","last_change":"2024-04-24T22:44:13.839383+0000","last_active":"2024-04-24T22:44:13.839259+0000","last_peered":"2024-04-24T22:44:13.839259+0000","last_clean":"2024-04-24T22:44:13.839259+0000","last_became_active":"2024-04-24T22:39:36.703461+0000","last_became_peered":"2024-04-24T22:39:36.703461+0000","last_unstale":"2024-04-24T22:44:13.839259+0000","last_undegraded":"2024-04-24T22:44:13.839259+0000","last_fullsized":"2024-04-24T22:44:13.839259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'2","last_scrub_stamp":"2024-04-24T22:44:13.839055+0000","last_deep_scrub":"21'2","last_deep_scrub_stamp":"2024-04-24T22:44:13.839055+0000","last_clean_scrub_stamp":"2024-04-24T22:44:13.839055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:40.467599+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'181","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:19.735699+0000","last_change":"2024-04-24T22:44:19.735764+0000","last_active":"2024-04-24T22:44:19.735699+0000","last_peered":"2024-04-24T22:44:19.735699+0000","last_clean":"2024-04-24T22:44:19.735699+0000","last_became_active":"2024-04-24T22:39:37.711704+0000","last_became_peered":"2024-04-24T22:39:37.711704+0000","last_unstale":"2024-04-24T22:44:19.735699+0000","last_undegraded":"2024-04-24T22:44:19.735699+0000","last_fullsized":"2024-04-24T22:44:19.735699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'181","last_scrub_stamp":"2024-04-24T22:44:19.735488+0000","last_deep_scrub":"23'181","last_deep_scrub_stamp":"2024-04-24T22:44:19.735488+0000","last_clean_scrub_stamp":"2024-04-24T22:44:19.735488+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:59.605411+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'19","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-24T22:44:13.296828+0000","last_change":"2024-04-24T22:44:13.296909+0000","last_active":"2024-04-24T22:44:13.296828+0000","last_peered":"2024-04-24T22:44:13.296828+0000","last_clean":"2024-04-24T22:44:13.296828+0000","last_became_active":"2024-04-24T22:39:36.715081+0000","last_became_peered":"2024-04-24T22:39:36.715081+0000","last_unstale":"2024-04-24T22:44:13.296828+0000","last_undegraded":"2024-04-24T22:44:13.296828+0000","last_fullsized":"2024-04-24T22:44:13.296828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-04-24T22:44:13.296668+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-04-24T22:44:13.296668+0000","last_clean_scrub_stamp":"2024-04-24T22:44:13.296668+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:33.532971+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11255,"ondisk_log_size":11255,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":391405,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":422,"num_write_kb":588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032192,"data_stored":782810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":472,"ondisk_log_size":472,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40348,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331492,"statfs":{"total":96636764160,"available":96595447808,"internally_reserved":0,"allocated":872448,"data_stored":680263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":651264,"data_stored":472664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":290816,"data_stored":92126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39756,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332084,"statfs":{"total":96636764160,"available":96596054016,"internally_reserved":0,"allocated":266240,"data_stored":89125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":659456,"data_stored":473203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39768,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332072,"statfs":{"total":96636764160,"available":96596041728,"internally_reserved":0,"allocated":278528,"data_stored":89762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39760,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332080,"statfs":{"total":96636764160,"available":96596049920,"internally_reserved":0,"allocated":270336,"data_stored":89726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":864256,"data_stored":679037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":385408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":384869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T22:44:37.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-24T22:44:37.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.806+0000 7fb087e00640 1 Processor -- start 2024-04-24T22:44:37.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.806+0000 7fb087e00640 1 -- start start 2024-04-24T22:44:37.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb087e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb088073040 0x7fb0881411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb087e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb08806bb90 0x7fb088141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb087e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb08815c3d0 con 0x7fb088072c60 2024-04-24T22:44:37.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb087e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb088158bf0 con 0x7fb08806bb90 2024-04-24T22:44:37.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb087e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb08815c6b0 con 0x7fb088073040 2024-04-24T22:44:37.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb086a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb088072c60 0x7fb0880c2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:43192/0 (socket says 172.21.15.47:43192) 2024-04-24T22:44:37.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb087400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb08806bb90 0x7fb088141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb086a00640 1 -- 172.21.15.47:0/2267564077 learned_addr learned my addr 172.21.15.47:0/2267564077 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:37.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb086000640 1 --2- 172.21.15.47:0/2267564077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb088073040 0x7fb0881411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb087400640 1 -- 172.21.15.47:0/2267564077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb088073040 msgr2=0x7fb0881411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:37.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb087400640 1 --2- 172.21.15.47:0/2267564077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb088073040 0x7fb0881411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb087400640 1 -- 172.21.15.47:0/2267564077 >> v1:172.21.15.113:6789/0 conn(0x7fb088072c60 legacy=0x7fb0880c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:37.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb087400640 1 -- 172.21.15.47:0/2267564077 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0881585a0 con 0x7fb08806bb90 2024-04-24T22:44:37.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.808+0000 7fb085600640 1 -- 172.21.15.47:0/2267564077 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832547137 0 0) 0x7fb08815c3d0 con 0x7fb088072c60 2024-04-24T22:44:37.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.810+0000 7fb087400640 1 --2- 172.21.15.47:0/2267564077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08806bb90 0x7fb088141730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb080009d90 tx=0x7fb08002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c86bf4c1ad772281 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:37.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.810+0000 7fb085600640 1 -- 172.21.15.47:0/2267564077 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080007740 con 0x7fb08806bb90 2024-04-24T22:44:37.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.810+0000 7fb085600640 1 -- 172.21.15.47:0/2267564077 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0800078e0 con 0x7fb08806bb90 2024-04-24T22:44:37.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.810+0000 7fb085600640 1 -- 172.21.15.47:0/2267564077 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080007cb0 con 0x7fb08806bb90 2024-04-24T22:44:37.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.810+0000 7fb087e00640 1 -- 172.21.15.47:0/2267564077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08806bb90 msgr2=0x7fb088141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:37.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.810+0000 7fb087e00640 1 --2- 172.21.15.47:0/2267564077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08806bb90 0x7fb088141730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb080009d90 tx=0x7fb08002a6f0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.811+0000 7fb087e00640 1 -- 172.21.15.47:0/2267564077 shutdown_connections 2024-04-24T22:44:37.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.811+0000 7fb087e00640 1 --2- 172.21.15.47:0/2267564077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08806bb90 0x7fb088141730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.811+0000 7fb087e00640 1 --2- 172.21.15.47:0/2267564077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb088073040 0x7fb0881411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.811+0000 7fb087e00640 1 -- 172.21.15.47:0/2267564077 >> 172.21.15.47:0/2267564077 conn(0x7fb0880cb900 msgr2=0x7fb0880cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:37.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.811+0000 7fb087e00640 1 -- 172.21.15.47:0/2267564077 shutdown_connections 2024-04-24T22:44:37.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.811+0000 7fb087e00640 1 -- 172.21.15.47:0/2267564077 wait complete. 2024-04-24T22:44:37.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.812+0000 7fb087e00640 1 Processor -- start 2024-04-24T22:44:37.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.812+0000 7fb087e00640 1 -- start start 2024-04-24T22:44:37.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.813+0000 7fb087e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08806bb90 0x7fb08813c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.813+0000 7fb087e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb088072c60 0x7fb088154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.813+0000 7fb087e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088073040 0x7fb08815b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.813+0000 7fb087e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb088159080 con 0x7fb088073040 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.813+0000 7fb087e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb088158270 con 0x7fb088072c60 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.813+0000 7fb087e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb088159200 con 0x7fb08806bb90 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb088072c60 0x7fb088154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08806bb90 0x7fb08813c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb088072c60 0x7fb088154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:57714/0 (socket says 172.21.15.47:57714) 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb087400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088073040 0x7fb08815b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086000640 1 -- 172.21.15.47:0/3576394145 learned_addr learned my addr 172.21.15.47:0/3576394145 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086a00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08806bb90 0x7fb08813c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:43566/0 (socket says 172.21.15.47:43566) 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086000640 1 -- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08806bb90 msgr2=0x7fb08813c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086000640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08806bb90 0x7fb08813c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086000640 1 -- 172.21.15.47:0/3576394145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088073040 msgr2=0x7fb08815b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:37.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086000640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088073040 0x7fb08815b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:37.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086000640 1 -- 172.21.15.47:0/3576394145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb088153840 con 0x7fb088072c60 2024-04-24T22:44:37.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086a00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08806bb90 0x7fb08813c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:37.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.814+0000 7fb086000640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb088072c60 0x7fb088154ab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb07000e870 tx=0x7fb07000ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:37.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.815+0000 7fb07ea00640 1 -- 172.21.15.47:0/3576394145 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070017070 con 0x7fb088072c60 2024-04-24T22:44:37.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.815+0000 7fb07ea00640 1 -- 172.21.15.47:0/3576394145 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07000c940 con 0x7fb088072c60 2024-04-24T22:44:37.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.815+0000 7fb07ea00640 1 -- 172.21.15.47:0/3576394145 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07000cd30 con 0x7fb088072c60 2024-04-24T22:44:37.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.815+0000 7fb087e00640 1 -- 172.21.15.47:0/3576394145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb088153b20 con 0x7fb088072c60 2024-04-24T22:44:37.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.815+0000 7fb087e00640 1 -- 172.21.15.47:0/3576394145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb088071880 con 0x7fb088072c60 2024-04-24T22:44:37.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.816+0000 7fb087e00640 1 -- 172.21.15.47:0/3576394145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb088068d50 con 0x7fb088072c60 2024-04-24T22:44:37.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.817+0000 7fb07ea00640 1 -- 172.21.15.47:0/3576394145 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb07001a070 con 0x7fb088072c60 2024-04-24T22:44:37.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.818+0000 7fb07ea00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb06406be60 0x7fb06406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:37.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.818+0000 7fb07ea00640 1 -- 172.21.15.47:0/3576394145 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb07008b100 con 0x7fb088072c60 2024-04-24T22:44:37.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.818+0000 7fb086a00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb06406be60 0x7fb06406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:37.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.819+0000 7fb086a00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb06406be60 0x7fb06406e320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb078000f60 tx=0x7fb078007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:37.822+0000 7fb07ea00640 1 -- 172.21.15.47:0/3576394145 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb070059340 con 0x7fb088072c60 2024-04-24T22:44:38.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.018+0000 7fb087e00640 1 -- 172.21.15.47:0/3576394145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fb088153cf0 con 0x7fb088072c60 2024-04-24T22:44:38.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.024+0000 7fb07ea00640 1 -- 172.21.15.47:0/3576394145 <== mon.1 v2:172.21.15.113:3300/0 7 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb07005d640 con 0x7fb088072c60 2024-04-24T22:44:38.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.026+0000 7fb07ea00640 1 -- 172.21.15.47:0/3576394145 <== mon.1 v2:172.21.15.113:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7fb070061e70 con 0x7fb088072c60 2024-04-24T22:44:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 -- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb06406be60 msgr2=0x7fb06406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb06406be60 0x7fb06406e320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb078000f60 tx=0x7fb078007d50 comp rx=0 tx=0).stop 2024-04-24T22:44:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 -- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb088072c60 msgr2=0x7fb088154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb088072c60 0x7fb088154ab0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb07000e870 tx=0x7fb07000ec70 comp rx=0 tx=0).stop 2024-04-24T22:44:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 -- 172.21.15.47:0/3576394145 shutdown_connections 2024-04-24T22:44:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fb06406be60 0x7fb06406e320 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb088073040 0x7fb08815b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb088072c60 0x7fb088154ab0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 --2- 172.21.15.47:0/3576394145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08806bb90 0x7fb08813c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 -- 172.21.15.47:0/3576394145 >> 172.21.15.47:0/3576394145 conn(0x7fb0880cb900 msgr2=0x7fb0880cb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 -- 172.21.15.47:0/3576394145 shutdown_connections 2024-04-24T22:44:38.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.029+0000 7fb087e00640 1 -- 172.21.15.47:0/3576394145 wait complete. 2024-04-24T22:44:38.050 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T22:44:38.050 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:44:38.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.254+0000 7f59fec00640 1 Processor -- start 2024-04-24T22:44:38.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.255+0000 7f59fec00640 1 -- start start 2024-04-24T22:44:38.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.256+0000 7f59fec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f80c7750 0x7f59f80c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.257+0000 7f59fec00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f80c7bb0 con 0x7f59f8152960 2024-04-24T22:44:38.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.257+0000 7f59fec00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f80c7e50 con 0x7f59f8072b80 2024-04-24T22:44:38.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.257+0000 7f59fec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f59f8158f20 con 0x7f59f80c7750 2024-04-24T22:44:38.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.257+0000 7f59f7e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f59f8152960 0x7f59f815ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51562/0 (socket says 172.21.15.47:51562) 2024-04-24T22:44:38.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.257+0000 7f59f7e00640 1 -- 172.21.15.47:0/3924180739 learned_addr learned my addr 172.21.15.47:0/3924180739 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:38.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.257+0000 7f59f7400640 1 --2- 172.21.15.47:0/3924180739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f80c7750 0x7f59f80c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.257+0000 7f59f6a00640 1 -- 172.21.15.47:0/3924180739 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1381047222 0 0) 0x7f59f80c7bb0 con 0x7f59f8152960 2024-04-24T22:44:38.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.257+0000 7f59f7400640 1 -- 172.21.15.47:0/3924180739 >> v1:172.21.15.113:6789/0 conn(0x7f59f8072b80 legacy=0x7f59f8162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.257+0000 7f59f7400640 1 -- 172.21.15.47:0/3924180739 >> v1:172.21.15.47:6789/0 conn(0x7f59f8152960 legacy=0x7f59f815ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.258+0000 7f59f7400640 1 -- 172.21.15.47:0/3924180739 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f81585f0 con 0x7f59f80c7750 2024-04-24T22:44:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.259+0000 7f59f7400640 1 --2- 172.21.15.47:0/3924180739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f80c7750 0x7f59f80c8fa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59f81590a0 tx=0x7f59e402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34b2488a76a890e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.259+0000 7f59f6a00640 1 -- 172.21.15.47:0/3924180739 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4037070 con 0x7f59f80c7750 2024-04-24T22:44:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.259+0000 7f59f6a00640 1 -- 172.21.15.47:0/3924180739 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f59e402ae40 con 0x7f59f80c7750 2024-04-24T22:44:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.259+0000 7f59f6a00640 1 -- 172.21.15.47:0/3924180739 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4032680 con 0x7f59f80c7750 2024-04-24T22:44:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.259+0000 7f59fec00640 1 -- 172.21.15.47:0/3924180739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f80c7750 msgr2=0x7f59f80c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.259+0000 7f59fec00640 1 --2- 172.21.15.47:0/3924180739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f80c7750 0x7f59f80c8fa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f59f81590a0 tx=0x7f59e402a130 comp rx=0 tx=0).stop 2024-04-24T22:44:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.259+0000 7f59fec00640 1 -- 172.21.15.47:0/3924180739 shutdown_connections 2024-04-24T22:44:38.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.259+0000 7f59fec00640 1 --2- 172.21.15.47:0/3924180739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f80c7750 0x7f59f80c8fa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.260+0000 7f59fec00640 1 -- 172.21.15.47:0/3924180739 >> 172.21.15.47:0/3924180739 conn(0x7f59f806e580 msgr2=0x7f59f806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:38.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.260+0000 7f59fec00640 1 -- 172.21.15.47:0/3924180739 shutdown_connections 2024-04-24T22:44:38.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.260+0000 7f59fec00640 1 -- 172.21.15.47:0/3924180739 wait complete. 2024-04-24T22:44:38.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.261+0000 7f59fec00640 1 Processor -- start 2024-04-24T22:44:38.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.261+0000 7f59fec00640 1 -- start start 2024-04-24T22:44:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.262+0000 7f59fec00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f8072b80 0x7f59f8070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.262+0000 7f59fec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f80c7750 0x7f59f80cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.262+0000 7f59fec00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f8152960 0x7f59f814e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.262+0000 7f59fec00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f59f80c8130 con 0x7f59f8072b80 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.262+0000 7f59fec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f59f81593b0 con 0x7f59f80c7750 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.262+0000 7f59fec00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f59f8158bb0 con 0x7f59f8152960 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.263+0000 7f59f7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f8072b80 0x7f59f8070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.263+0000 7f59f7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f8072b80 0x7f59f8070ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42204/0 (socket says 172.21.15.47:42204) 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.263+0000 7f59f7e00640 1 -- 172.21.15.47:0/761071087 learned_addr learned my addr 172.21.15.47:0/761071087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.263+0000 7f59f7400640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f80c7750 0x7f59f80cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.263+0000 7f59fcc00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f8152960 0x7f59f814e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.263+0000 7f59f7e00640 1 -- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f8152960 msgr2=0x7f59f814e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.263+0000 7f59f7e00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f8152960 0x7f59f814e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.263+0000 7f59f7e00640 1 -- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f80c7750 msgr2=0x7f59f80cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.263+0000 7f59f7e00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f80c7750 0x7f59f80cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.263+0000 7f59f7e00640 1 -- 172.21.15.47:0/761071087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f81f07e0 con 0x7f59f8072b80 2024-04-24T22:44:38.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.264+0000 7f59f7e00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f8072b80 0x7f59f8070ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59e800c8f0 tx=0x7f59e800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.264+0000 7f59efe00640 1 -- 172.21.15.47:0/761071087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8018070 con 0x7f59f8072b80 2024-04-24T22:44:38.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.264+0000 7f59efe00640 1 -- 172.21.15.47:0/761071087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f59e800a640 con 0x7f59f8072b80 2024-04-24T22:44:38.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.264+0000 7f59fec00640 1 -- 172.21.15.47:0/761071087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f80c4840 con 0x7f59f8072b80 2024-04-24T22:44:38.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.264+0000 7f59efe00640 1 -- 172.21.15.47:0/761071087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e800aa50 con 0x7f59f8072b80 2024-04-24T22:44:38.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.264+0000 7f59fec00640 1 -- 172.21.15.47:0/761071087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f81f0aa0 con 0x7f59f8072b80 2024-04-24T22:44:38.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.265+0000 7f59fec00640 1 -- 172.21.15.47:0/761071087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f81f1090 con 0x7f59f8072b80 2024-04-24T22:44:38.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.266+0000 7f59efe00640 1 -- 172.21.15.47:0/761071087 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59e801b070 con 0x7f59f8072b80 2024-04-24T22:44:38.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.267+0000 7f59efe00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f59d806be60 0x7f59d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.267+0000 7f59efe00640 1 -- 172.21.15.47:0/761071087 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f59e808a5d0 con 0x7f59f8072b80 2024-04-24T22:44:38.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.268+0000 7f59f7400640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f59d806be60 0x7f59d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.268+0000 7f59f7400640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f59d806be60 0x7f59d806e320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59e4034000 tx=0x7f59e402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.272+0000 7f59efe00640 1 -- 172.21.15.47:0/761071087 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f59e8058780 con 0x7f59f8072b80 2024-04-24T22:44:38.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.515+0000 7f59fec00640 1 -- 172.21.15.47:0/761071087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59f8073b90 con 0x7f59f8072b80 2024-04-24T22:44:38.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.516+0000 7f59efe00640 1 -- 172.21.15.47:0/761071087 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6916 (secure 0 0 0) 0x7f59e805ca80 con 0x7f59f8072b80 2024-04-24T22:44:38.517 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:38.518 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1712577517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1712577517},{"type":"v1","addr":"172.21.15.113:6839","nonce":1712577517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:39:37.178621+0000","modified":"2024-04-24T22:40:06.837213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4282,"mds_2":4279,"mds_3":4241,"mds_4":4238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4238":{"gid":4238,"name":"a","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6833/2908847255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2908847255},{"type":"v1","addr":"172.21.15.47:6833","nonce":2908847255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.47:6837/2173854280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2173854280},{"type":"v1","addr":"172.21.15.47:6837","nonce":2173854280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/3103107210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3103107210},{"type":"v1","addr":"172.21.15.113:6837","nonce":3103107210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4282":{"gid":4282,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6838/3621458364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3621458364},{"type":"v1","addr":"172.21.15.113:6838","nonce":3621458364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.47:6835/1117134154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1117134154},{"type":"v1","addr":"172.21.15.47:6835","nonce":1117134154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4238,4241,4306,4279,4282]},"id":1}]} 2024-04-24T22:44:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2024-04-24T22:44:38.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.518+0000 7f59fec00640 1 -- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f59d806be60 msgr2=0x7f59d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.518+0000 7f59fec00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f59d806be60 0x7f59d806e320 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59e4034000 tx=0x7f59e402e000 comp rx=0 tx=0).stop 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.519+0000 7f59fec00640 1 -- 172.21.15.47:0/761071087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f8072b80 msgr2=0x7f59f8070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.519+0000 7f59fec00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f8072b80 0x7f59f8070ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59e800c8f0 tx=0x7f59e800cdb0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.519+0000 7f59fec00640 1 -- 172.21.15.47:0/761071087 shutdown_connections 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.519+0000 7f59fec00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f59d806be60 0x7f59d806e320 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.519+0000 7f59fec00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59f8152960 0x7f59f814e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.519+0000 7f59fec00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f80c7750 0x7f59f80cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.519+0000 7f59fec00640 1 --2- 172.21.15.47:0/761071087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f8072b80 0x7f59f8070ef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.519+0000 7f59fec00640 1 -- 172.21.15.47:0/761071087 >> 172.21.15.47:0/761071087 conn(0x7f59f806e580 msgr2=0x7f59f81508e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.519+0000 7f59fec00640 1 -- 172.21.15.47:0/761071087 shutdown_connections 2024-04-24T22:44:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.519+0000 7f59fec00640 1 -- 172.21.15.47:0/761071087 wait complete. 2024-04-24T22:44:38.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T22:44:38.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.743+0000 7fa8f9600640 1 Processor -- start 2024-04-24T22:44:38.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.743+0000 7fa8f9600640 1 -- start start 2024-04-24T22:44:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.745+0000 7fa8f9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8f406e1b0 0x7fa8f406e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.745+0000 7fa8f9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f415e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.745+0000 7fa8f9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8f40c6f30 con 0x7fa8f4070ea0 2024-04-24T22:44:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.745+0000 7fa8f9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f40c7210 con 0x7fa8f406e1b0 2024-04-24T22:44:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.745+0000 7fa8f9600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f415a500 con 0x7fa8f40707e0 2024-04-24T22:44:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.745+0000 7fa8ebe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f415e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.745+0000 7fa8f3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8f406e1b0 0x7fa8f406e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.745+0000 7fa8f8c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa8f4070ea0 0x7fa8f406b160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:43210/0 (socket says 172.21.15.47:43210) 2024-04-24T22:44:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.745+0000 7fa8f8c00640 1 -- 172.21.15.47:0/3121804860 learned_addr learned my addr 172.21.15.47:0/3121804860 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:38.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.746+0000 7fa8ebe00640 1 -- 172.21.15.47:0/3121804860 >> v2:172.21.15.113:3300/0 conn(0x7fa8f406e1b0 msgr2=0x7fa8f406e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:38.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.746+0000 7fa8ebe00640 1 --2- 172.21.15.47:0/3121804860 >> v2:172.21.15.113:3300/0 conn(0x7fa8f406e1b0 0x7fa8f406e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.746+0000 7fa8ebe00640 1 -- 172.21.15.47:0/3121804860 >> v1:172.21.15.113:6789/0 conn(0x7fa8f4070ea0 legacy=0x7fa8f406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.746+0000 7fa8ebe00640 1 -- 172.21.15.47:0/3121804860 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f415a7e0 con 0x7fa8f40707e0 2024-04-24T22:44:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.747+0000 7fa8ebe00640 1 --2- 172.21.15.47:0/3121804860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f415e3a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0097a0 tx=0x7fa8dc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d4a277e09a69cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.747+0000 7fa8f3400640 1 -- 172.21.15.47:0/3121804860 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc037070 con 0x7fa8f40707e0 2024-04-24T22:44:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.747+0000 7fa8f3400640 1 -- 172.21.15.47:0/3121804860 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa8dc032440 con 0x7fa8f40707e0 2024-04-24T22:44:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.747+0000 7fa8f3400640 1 -- 172.21.15.47:0/3121804860 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc032830 con 0x7fa8f40707e0 2024-04-24T22:44:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.747+0000 7fa8f9600640 1 -- 172.21.15.47:0/3121804860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 msgr2=0x7fa8f415e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.748+0000 7fa8f9600640 1 --2- 172.21.15.47:0/3121804860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f415e3a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0097a0 tx=0x7fa8dc02a180 comp rx=0 tx=0).stop 2024-04-24T22:44:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.748+0000 7fa8f9600640 1 -- 172.21.15.47:0/3121804860 shutdown_connections 2024-04-24T22:44:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.748+0000 7fa8f9600640 1 --2- 172.21.15.47:0/3121804860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f415e3a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.748+0000 7fa8f9600640 1 --2- 172.21.15.47:0/3121804860 >> v2:172.21.15.113:3300/0 conn(0x7fa8f406e1b0 0x7fa8f406e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.748+0000 7fa8f9600640 1 -- 172.21.15.47:0/3121804860 >> 172.21.15.47:0/3121804860 conn(0x7fa8f40cb900 msgr2=0x7fa8f40cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.748+0000 7fa8f9600640 1 -- 172.21.15.47:0/3121804860 shutdown_connections 2024-04-24T22:44:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.748+0000 7fa8f9600640 1 -- 172.21.15.47:0/3121804860 wait complete. 2024-04-24T22:44:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.750+0000 7fa8f9600640 1 Processor -- start 2024-04-24T22:44:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.750+0000 7fa8f9600640 1 -- start start 2024-04-24T22:44:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8f9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8f406e1b0 0x7fa8f4159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8f9600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f41e3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8f9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8f4070ea0 0x7fa8f414f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8f9600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f415b2c0 con 0x7fa8f406e1b0 2024-04-24T22:44:38.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8f9600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f415e8e0 con 0x7fa8f4070ea0 2024-04-24T22:44:38.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8f9600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f4154950 con 0x7fa8f40707e0 2024-04-24T22:44:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8f406e1b0 0x7fa8f4159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8ebe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f41e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8f8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8f4070ea0 0x7fa8f414f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8ebe00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f41e3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:43598/0 (socket says 172.21.15.47:43598) 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8f406e1b0 0x7fa8f4159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42210/0 (socket says 172.21.15.47:42210) 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.751+0000 7fa8ebe00640 1 -- 172.21.15.47:0/3728548663 learned_addr learned my addr 172.21.15.47:0/3728548663 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8ebe00640 1 -- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8f4070ea0 msgr2=0x7fa8f414f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8ebe00640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8f4070ea0 0x7fa8f414f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8ebe00640 1 -- 172.21.15.47:0/3728548663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8f406e1b0 msgr2=0x7fa8f4159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8ebe00640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8f406e1b0 0x7fa8f4159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8ebe00640 1 -- 172.21.15.47:0/3728548663 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f4156ba0 con 0x7fa8f40707e0 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8f3e00640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8f406e1b0 0x7fa8f4159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8ebe00640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f41e3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc000c00 tx=0x7fa8dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8f0c00640 1 -- 172.21.15.47:0/3728548663 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc041030 con 0x7fa8f40707e0 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8f0c00640 1 -- 172.21.15.47:0/3728548663 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa8dc030540 con 0x7fa8f40707e0 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8f9600640 1 -- 172.21.15.47:0/3728548663 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f4156e80 con 0x7fa8f40707e0 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.752+0000 7fa8f0c00640 1 -- 172.21.15.47:0/3728548663 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc037040 con 0x7fa8f40707e0 2024-04-24T22:44:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.753+0000 7fa8f9600640 1 -- 172.21.15.47:0/3728548663 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f41f0d60 con 0x7fa8f40707e0 2024-04-24T22:44:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.754+0000 7fa8f9600640 1 -- 172.21.15.47:0/3728548663 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f4157050 con 0x7fa8f40707e0 2024-04-24T22:44:38.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.755+0000 7fa8f0c00640 1 -- 172.21.15.47:0/3728548663 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8dc044070 con 0x7fa8f40707e0 2024-04-24T22:44:38.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.756+0000 7fa8f0c00640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa8d006be80 0x7fa8d006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:38.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.756+0000 7fa8f0c00640 1 -- 172.21.15.47:0/3728548663 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa8dc0ab3e0 con 0x7fa8f40707e0 2024-04-24T22:44:38.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.756+0000 7fa8f3e00640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa8d006be80 0x7fa8d006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:38.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.757+0000 7fa8f3e00640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa8d006be80 0x7fa8d006e340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009e90 tx=0x7fa8e4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:38.760+0000 7fa8f0c00640 1 -- 172.21.15.47:0/3728548663 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa8f4157050 con 0x7fa8f40707e0 2024-04-24T22:44:39.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.006+0000 7fa8f9600640 1 -- 172.21.15.47:0/3728548663 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8f41577a0 con 0x7fa8f40707e0 2024-04-24T22:44:39.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.008+0000 7fa8f0c00640 1 -- 172.21.15.47:0/3728548663 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6916 (secure 0 0 0) 0x7fa8dc079730 con 0x7fa8f40707e0 2024-04-24T22:44:39.009 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:39.010 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1712577517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1712577517},{"type":"v1","addr":"172.21.15.113:6839","nonce":1712577517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T22:39:37.178621+0000","modified":"2024-04-24T22:40:06.837213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4306,"mds_1":4282,"mds_2":4279,"mds_3":4241,"mds_4":4238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4238":{"gid":4238,"name":"a","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6833/2908847255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2908847255},{"type":"v1","addr":"172.21.15.47:6833","nonce":2908847255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.47:6837/2173854280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2173854280},{"type":"v1","addr":"172.21.15.47:6837","nonce":2173854280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/3103107210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3103107210},{"type":"v1","addr":"172.21.15.113:6837","nonce":3103107210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4282":{"gid":4282,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6838/3621458364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3621458364},{"type":"v1","addr":"172.21.15.113:6838","nonce":3621458364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4306":{"gid":4306,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.47:6835/1117134154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1117134154},{"type":"v1","addr":"172.21.15.47:6835","nonce":1117134154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4238,4241,4306,4279,4282]},"id":1}]} 2024-04-24T22:44:39.010 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2024-04-24T22:44:39.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.010+0000 7fa8f9600640 1 -- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa8d006be80 msgr2=0x7fa8d006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.010+0000 7fa8f9600640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa8d006be80 0x7fa8d006e340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009e90 tx=0x7fa8e4009190 comp rx=0 tx=0).stop 2024-04-24T22:44:39.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.010+0000 7fa8f9600640 1 -- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 msgr2=0x7fa8f41e3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.010+0000 7fa8f9600640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f41e3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc000c00 tx=0x7fa8dc0046c0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.010+0000 7fa8f9600640 1 -- 172.21.15.47:0/3728548663 shutdown_connections 2024-04-24T22:44:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.010+0000 7fa8f9600640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fa8d006be80 0x7fa8d006e340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.010+0000 7fa8f9600640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8f4070ea0 0x7fa8f414f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.010+0000 7fa8f9600640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8f40707e0 0x7fa8f41e3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.010+0000 7fa8f9600640 1 --2- 172.21.15.47:0/3728548663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8f406e1b0 0x7fa8f4159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.010+0000 7fa8f9600640 1 -- 172.21.15.47:0/3728548663 >> 172.21.15.47:0/3728548663 conn(0x7fa8f40cb900 msgr2=0x7fa8f40cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.011+0000 7fa8f9600640 1 -- 172.21.15.47:0/3728548663 shutdown_connections 2024-04-24T22:44:39.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.011+0000 7fa8f9600640 1 -- 172.21.15.47:0/3728548663 wait complete. 2024-04-24T22:44:39.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T22:44:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.237+0000 7f2061e00640 1 Processor -- start 2024-04-24T22:44:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.237+0000 7f2061e00640 1 -- start start 2024-04-24T22:44:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.238+0000 7f2061e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f205c072c60 0x7f205c0c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.238+0000 7f2061e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c073040 0x7f205c1411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.238+0000 7f2061e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f205c15c3d0 con 0x7f205c06bb90 2024-04-24T22:44:39.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.238+0000 7f2061e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f205c158bf0 con 0x7f205c072c60 2024-04-24T22:44:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.238+0000 7f2061e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f205c15c6b0 con 0x7f205c073040 2024-04-24T22:44:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f2061400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f205c06bb90 0x7f205c15b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51564/0 (socket says 172.21.15.47:51564) 2024-04-24T22:44:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f2061400640 1 -- 172.21.15.47:0/2991572941 learned_addr learned my addr 172.21.15.47:0/2991572941 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f205be00640 1 --2- 172.21.15.47:0/2991572941 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c073040 0x7f205c1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f2060a00640 1 --2- 172.21.15.47:0/2991572941 >> v2:172.21.15.113:3300/0 conn(0x7f205c072c60 0x7f205c0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f205b400640 1 -- 172.21.15.47:0/2991572941 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983613215 0 0) 0x7f205c15c3d0 con 0x7f205c06bb90 2024-04-24T22:44:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f205b400640 1 -- 172.21.15.47:0/2991572941 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2044003520 con 0x7f205c06bb90 2024-04-24T22:44:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f205be00640 1 -- 172.21.15.47:0/2991572941 >> v2:172.21.15.113:3300/0 conn(0x7f205c072c60 msgr2=0x7f205c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f205be00640 1 --2- 172.21.15.47:0/2991572941 >> v2:172.21.15.113:3300/0 conn(0x7f205c072c60 0x7f205c0c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f205be00640 1 -- 172.21.15.47:0/2991572941 >> v1:172.21.15.47:6789/0 conn(0x7f205c06bb90 legacy=0x7f205c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f205be00640 1 -- 172.21.15.47:0/2991572941 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c1585a0 con 0x7f205c073040 2024-04-24T22:44:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.239+0000 7f2060a00640 1 --2- 172.21.15.47:0/2991572941 >> v2:172.21.15.113:3300/0 conn(0x7f205c072c60 0x7f205c0c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.240+0000 7f205be00640 1 --2- 172.21.15.47:0/2991572941 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c073040 0x7f205c1411f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20480097a0 tx=0x7f204802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a35642c45e4ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.241+0000 7f205b400640 1 -- 172.21.15.47:0/2991572941 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048037070 con 0x7f205c073040 2024-04-24T22:44:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.241+0000 7f205b400640 1 -- 172.21.15.47:0/2991572941 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2048032440 con 0x7f205c073040 2024-04-24T22:44:39.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.241+0000 7f205b400640 1 -- 172.21.15.47:0/2991572941 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048032830 con 0x7f205c073040 2024-04-24T22:44:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.241+0000 7f2061e00640 1 -- 172.21.15.47:0/2991572941 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c073040 msgr2=0x7f205c1411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.241+0000 7f2061e00640 1 --2- 172.21.15.47:0/2991572941 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c073040 0x7f205c1411f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20480097a0 tx=0x7f204802a210 comp rx=0 tx=0).stop 2024-04-24T22:44:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.241+0000 7f2061e00640 1 -- 172.21.15.47:0/2991572941 shutdown_connections 2024-04-24T22:44:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.242+0000 7f2061e00640 1 --2- 172.21.15.47:0/2991572941 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c073040 0x7f205c1411f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.242+0000 7f2061e00640 1 --2- 172.21.15.47:0/2991572941 >> v2:172.21.15.113:3300/0 conn(0x7f205c072c60 0x7f205c0c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.242+0000 7f2061e00640 1 -- 172.21.15.47:0/2991572941 >> 172.21.15.47:0/2991572941 conn(0x7f205c0cb900 msgr2=0x7f205c0cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.242+0000 7f2061e00640 1 -- 172.21.15.47:0/2991572941 shutdown_connections 2024-04-24T22:44:39.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.242+0000 7f2061e00640 1 -- 172.21.15.47:0/2991572941 wait complete. 2024-04-24T22:44:39.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.243+0000 7f2061e00640 1 Processor -- start 2024-04-24T22:44:39.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.243+0000 7f2061e00640 1 -- start start 2024-04-24T22:44:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.244+0000 7f2061e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06bb90 0x7f205c0c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.244+0000 7f2061e00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c072c60 0x7f205c0b9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.244+0000 7f2061e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f205c073040 0x7f205c13df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.244+0000 7f2061e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f205c0c32a0 con 0x7f205c073040 2024-04-24T22:44:39.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.244+0000 7f2061e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f205c158270 con 0x7f205c06bb90 2024-04-24T22:44:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.244+0000 7f2061e00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f205c159080 con 0x7f205c072c60 2024-04-24T22:44:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f2061400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f205c073040 0x7f205c13df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f2061400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f205c073040 0x7f205c13df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42218/0 (socket says 172.21.15.47:42218) 2024-04-24T22:44:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f2061400640 1 -- 172.21.15.47:0/1933511517 learned_addr learned my addr 172.21.15.47:0/1933511517 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f2060a00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06bb90 0x7f205c0c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f205be00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c072c60 0x7f205c0b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f2060a00640 1 -- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c072c60 msgr2=0x7f205c0b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f2060a00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c072c60 0x7f205c0b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f2060a00640 1 -- 172.21.15.47:0/1933511517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f205c073040 msgr2=0x7f205c13df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f2060a00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f205c073040 0x7f205c13df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f2060a00640 1 -- 172.21.15.47:0/1933511517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c0c7bb0 con 0x7f205c06bb90 2024-04-24T22:44:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.245+0000 7f205be00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c072c60 0x7f205c0b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:39.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.246+0000 7f2060a00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06bb90 0x7f205c0c7470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f205000e930 tx=0x7f205000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:39.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.246+0000 7f2058c00640 1 -- 172.21.15.47:0/1933511517 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050017070 con 0x7f205c06bb90 2024-04-24T22:44:39.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.246+0000 7f2058c00640 1 -- 172.21.15.47:0/1933511517 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f205000ca00 con 0x7f205c06bb90 2024-04-24T22:44:39.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.246+0000 7f2058c00640 1 -- 172.21.15.47:0/1933511517 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205000cdf0 con 0x7f205c06bb90 2024-04-24T22:44:39.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.246+0000 7f2061e00640 1 -- 172.21.15.47:0/1933511517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c0c7e90 con 0x7f205c06bb90 2024-04-24T22:44:39.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.246+0000 7f2061e00640 1 -- 172.21.15.47:0/1933511517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c0c8450 con 0x7f205c06bb90 2024-04-24T22:44:39.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.247+0000 7f2061e00640 1 -- 172.21.15.47:0/1933511517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c0c4dd0 con 0x7f205c06bb90 2024-04-24T22:44:39.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.248+0000 7f2058c00640 1 -- 172.21.15.47:0/1933511517 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f205001a070 con 0x7f205c06bb90 2024-04-24T22:44:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.249+0000 7f2058c00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f203c06be30 0x7f203c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.249+0000 7f2058c00640 1 -- 172.21.15.47:0/1933511517 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f205008aeb0 con 0x7f205c06bb90 2024-04-24T22:44:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.249+0000 7f205be00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f203c06be30 0x7f203c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.250+0000 7f205be00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f203c06be30 0x7f203c06e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2048034000 tx=0x7f204802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:39.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.253+0000 7f2058c00640 1 -- 172.21.15.47:0/1933511517 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f205c0c4dd0 con 0x7f205c06bb90 2024-04-24T22:44:39.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.455+0000 7f2061e00640 1 -- 172.21.15.47:0/1933511517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f205c0c3f70 con 0x7f205c06bb90 2024-04-24T22:44:39.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.457+0000 7f2058c00640 1 -- 172.21.15.47:0/1933511517 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16843 (secure 0 0 0) 0x7f205c0c3f70 con 0x7f205c06bb90 2024-04-24T22:44:39.459 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-24T22:44:39.459 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"fsid":"617410c3-1ae2-44fe-8492-d21ee235c3cc","created":"2024-04-24T22:39:18.524710+0000","modified":"2024-04-24T22:40:23.897411+0000","last_up_change":"2024-04-24T22:39:29.145276+0000","last_in_change":"2024-04-24T22:39:20.849444+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T22:39:30.711538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T22:39:34.803185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T22:39:36.184123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aadbbff1-8bdf-473f-93ef-bc88982ffdce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6809","nonce":2543493744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6811","nonce":2543493744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6815","nonce":2543493744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2543493744},{"type":"v1","addr":"172.21.15.47:6813","nonce":2543493744}]},"public_addr":"172.21.15.47:6809/2543493744","cluster_addr":"172.21.15.47:6811/2543493744","heartbeat_back_addr":"172.21.15.47:6815/2543493744","heartbeat_front_addr":"172.21.15.47:6813/2543493744","state":["exists","up"]},{"osd":1,"uuid":"4ade4f53-014c-4118-8f91-a4e15071760e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6801","nonce":623040628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6803","nonce":623040628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6807","nonce":623040628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":623040628},{"type":"v1","addr":"172.21.15.47:6805","nonce":623040628}]},"public_addr":"172.21.15.47:6801/623040628","cluster_addr":"172.21.15.47:6803/623040628","heartbeat_back_addr":"172.21.15.47:6807/623040628","heartbeat_front_addr":"172.21.15.47:6805/623040628","state":["exists","up"]},{"osd":2,"uuid":"252f4784-03e2-4b28-bb96-9ce9c33a40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6817","nonce":4187599812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6819","nonce":4187599812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6823","nonce":4187599812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4187599812},{"type":"v1","addr":"172.21.15.47:6821","nonce":4187599812}]},"public_addr":"172.21.15.47:6817/4187599812","cluster_addr":"172.21.15.47:6819/4187599812","heartbeat_back_addr":"172.21.15.47:6823/4187599812","heartbeat_front_addr":"172.21.15.47:6821/4187599812","state":["exists","up"]},{"osd":3,"uuid":"ee980f09-6a59-4d99-818e-431e9e755849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6825","nonce":872127764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6827","nonce":872127764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6831","nonce":872127764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":872127764},{"type":"v1","addr":"172.21.15.47:6829","nonce":872127764}]},"public_addr":"172.21.15.47:6825/872127764","cluster_addr":"172.21.15.47:6827/872127764","heartbeat_back_addr":"172.21.15.47:6831/872127764","heartbeat_front_addr":"172.21.15.47:6829/872127764","state":["exists","up"]},{"osd":4,"uuid":"9c4356d4-c36e-480c-bb3d-887f722e946c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6817","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6819","nonce":2737975939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6821","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6823","nonce":2737975939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6829","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6831","nonce":2737975939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6825","nonce":2737975939},{"type":"v1","addr":"172.21.15.113:6827","nonce":2737975939}]},"public_addr":"172.21.15.113:6819/2737975939","cluster_addr":"172.21.15.113:6823/2737975939","heartbeat_back_addr":"172.21.15.113:6831/2737975939","heartbeat_front_addr":"172.21.15.113:6827/2737975939","state":["exists","up"]},{"osd":5,"uuid":"55580504-e9dc-47e0-841b-3a022c2b3328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6801","nonce":1697310705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6803","nonce":1697310705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6807","nonce":1697310705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1697310705},{"type":"v1","addr":"172.21.15.113:6805","nonce":1697310705}]},"public_addr":"172.21.15.113:6801/1697310705","cluster_addr":"172.21.15.113:6803/1697310705","heartbeat_back_addr":"172.21.15.113:6807/1697310705","heartbeat_front_addr":"172.21.15.113:6805/1697310705","state":["exists","up"]},{"osd":6,"uuid":"a711f1d9-356f-43d9-b43e-849f4815402b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6809","nonce":1023357201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6811","nonce":1023357201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6815","nonce":1023357201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1023357201},{"type":"v1","addr":"172.21.15.113:6813","nonce":1023357201}]},"public_addr":"172.21.15.113:6809/1023357201","cluster_addr":"172.21.15.113:6811/1023357201","heartbeat_back_addr":"172.21.15.113:6815/1023357201","heartbeat_front_addr":"172.21.15.113:6813/1023357201","state":["exists","up"]},{"osd":7,"uuid":"cc5ddb38-cf05-42cd-b3d8-41f011a14345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6818","nonce":2022093233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6822","nonce":2022093233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6830","nonce":2022093233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2022093233},{"type":"v1","addr":"172.21.15.113:6826","nonce":2022093233}]},"public_addr":"172.21.15.113:6818/2022093233","cluster_addr":"172.21.15.113:6822/2022093233","heartbeat_back_addr":"172.21.15.113:6830/2022093233","heartbeat_front_addr":"172.21.15.113:6826/2022093233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.504811+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.229850+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.270146+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.552785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.007855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.012014+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T22:39:26.004149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T22:44:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 -- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f203c06be30 msgr2=0x7f203c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f203c06be30 0x7f203c06e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2048034000 tx=0x7f204802e000 comp rx=0 tx=0).stop 2024-04-24T22:44:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 -- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06bb90 msgr2=0x7f205c0c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06bb90 0x7f205c0c7470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f205000e930 tx=0x7f205000edf0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 -- 172.21.15.47:0/1933511517 shutdown_connections 2024-04-24T22:44:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f203c06be30 0x7f203c06e2f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f205c073040 0x7f205c13df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c072c60 0x7f205c0b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 --2- 172.21.15.47:0/1933511517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06bb90 0x7f205c0c7470 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 -- 172.21.15.47:0/1933511517 >> 172.21.15.47:0/1933511517 conn(0x7f205c0cb900 msgr2=0x7f205c0c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:39.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 -- 172.21.15.47:0/1933511517 shutdown_connections 2024-04-24T22:44:39.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.460+0000 7f2061e00640 1 -- 172.21.15.47:0/1933511517 wait complete. 2024-04-24T22:44:39.483 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T22:44:39.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.696+0000 7ff735200640 1 Processor -- start 2024-04-24T22:44:39.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.696+0000 7ff735200640 1 -- start start 2024-04-24T22:44:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.697+0000 7ff735200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff73006e1b0 0x7ff73006e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.697+0000 7ff735200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff730162d40 con 0x7ff7300c2850 2024-04-24T22:44:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.697+0000 7ff735200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff730161e70 con 0x7ff7300c6880 2024-04-24T22:44:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.697+0000 7ff735200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff7300c71f0 con 0x7ff73006e1b0 2024-04-24T22:44:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.697+0000 7ff72ea00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff7300c2850 0x7ff730162630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51576/0 (socket says 172.21.15.47:51576) 2024-04-24T22:44:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.697+0000 7ff72ea00640 1 -- 172.21.15.47:0/560819811 learned_addr learned my addr 172.21.15.47:0/560819811 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.698+0000 7ff72f400640 1 --2- 172.21.15.47:0/560819811 >> v2:172.21.15.113:3300/0 conn(0x7ff73006e1b0 0x7ff73006e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.698+0000 7ff72e000640 1 -- 172.21.15.47:0/560819811 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3031573323 0 0) 0x7ff71c001110 con 0x7ff7300c2850 2024-04-24T22:44:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.698+0000 7ff72e000640 1 -- 172.21.15.47:0/560819811 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff714003520 con 0x7ff7300c2850 2024-04-24T22:44:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.698+0000 7ff72f400640 1 -- 172.21.15.47:0/560819811 >> v1:172.21.15.113:6789/0 conn(0x7ff7300c6880 legacy=0x7ff730169160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.698+0000 7ff72e000640 1 -- 172.21.15.47:0/560819811 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3723327786 0 0) 0x7ff730161e70 con 0x7ff7300c6880 2024-04-24T22:44:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.698+0000 7ff72f400640 1 -- 172.21.15.47:0/560819811 >> v1:172.21.15.47:6789/0 conn(0x7ff7300c2850 legacy=0x7ff730162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.698+0000 7ff72f400640 1 -- 172.21.15.47:0/560819811 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7300c7810 con 0x7ff73006e1b0 2024-04-24T22:44:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.699+0000 7ff72f400640 1 --2- 172.21.15.47:0/560819811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff73006e1b0 0x7ff73006e590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff720009d90 tx=0x7ff72002a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f4ffcb87b71d89a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.700+0000 7ff72e000640 1 -- 172.21.15.47:0/560819811 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720037070 con 0x7ff73006e1b0 2024-04-24T22:44:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.700+0000 7ff72e000640 1 -- 172.21.15.47:0/560819811 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff720007740 con 0x7ff73006e1b0 2024-04-24T22:44:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.700+0000 7ff72e000640 1 -- 172.21.15.47:0/560819811 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720007b30 con 0x7ff73006e1b0 2024-04-24T22:44:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.700+0000 7ff735200640 1 -- 172.21.15.47:0/560819811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff73006e1b0 msgr2=0x7ff73006e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.700+0000 7ff735200640 1 --2- 172.21.15.47:0/560819811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff73006e1b0 0x7ff73006e590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff720009d90 tx=0x7ff72002a780 comp rx=0 tx=0).stop 2024-04-24T22:44:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.700+0000 7ff735200640 1 -- 172.21.15.47:0/560819811 shutdown_connections 2024-04-24T22:44:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.700+0000 7ff735200640 1 --2- 172.21.15.47:0/560819811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff73006e1b0 0x7ff73006e590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.700+0000 7ff735200640 1 -- 172.21.15.47:0/560819811 >> 172.21.15.47:0/560819811 conn(0x7ff7300cb900 msgr2=0x7ff7300cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.700+0000 7ff735200640 1 -- 172.21.15.47:0/560819811 shutdown_connections 2024-04-24T22:44:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.700+0000 7ff735200640 1 -- 172.21.15.47:0/560819811 wait complete. 2024-04-24T22:44:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.702+0000 7ff735200640 1 Processor -- start 2024-04-24T22:44:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.702+0000 7ff735200640 1 -- start start 2024-04-24T22:44:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff735200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff73006e1b0 0x7ff730144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff735200640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7300c2850 0x7ff73014c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff735200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7300c6880 0x7ff73014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff735200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff73015fcd0 con 0x7ff73006e1b0 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff735200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff7300c7dd0 con 0x7ff7300c6880 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff735200640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff730158e00 con 0x7ff7300c2850 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff72f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff73006e1b0 0x7ff730144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff72fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7300c6880 0x7ff73014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff72f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff73006e1b0 0x7ff730144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42222/0 (socket says 172.21.15.47:42222) 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff72f400640 1 -- 172.21.15.47:0/967185212 learned_addr learned my addr 172.21.15.47:0/967185212 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.703+0000 7ff72ea00640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7300c2850 0x7ff73014c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff72fe00640 1 -- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7300c2850 msgr2=0x7ff73014c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff72fe00640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7300c2850 0x7ff73014c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff72fe00640 1 -- 172.21.15.47:0/967185212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff73006e1b0 msgr2=0x7ff730144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff72fe00640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff73006e1b0 0x7ff730144460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff72fe00640 1 -- 172.21.15.47:0/967185212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7301f5c10 con 0x7ff7300c6880 2024-04-24T22:44:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff72f400640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff73006e1b0 0x7ff730144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff72fe00640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7300c6880 0x7ff73014cc50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff72800c410 tx=0x7ff72800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff727400640 1 -- 172.21.15.47:0/967185212 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728017070 con 0x7ff7300c6880 2024-04-24T22:44:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff727400640 1 -- 172.21.15.47:0/967185212 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff7280035f0 con 0x7ff7300c6880 2024-04-24T22:44:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff727400640 1 -- 172.21.15.47:0/967185212 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728003a00 con 0x7ff7300c6880 2024-04-24T22:44:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.704+0000 7ff735200640 1 -- 172.21.15.47:0/967185212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7300cb190 con 0x7ff7300c6880 2024-04-24T22:44:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.705+0000 7ff735200640 1 -- 172.21.15.47:0/967185212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7300cb450 con 0x7ff7300c6880 2024-04-24T22:44:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.706+0000 7ff735200640 1 -- 172.21.15.47:0/967185212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7301f5a40 con 0x7ff7300c6880 2024-04-24T22:44:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.707+0000 7ff727400640 1 -- 172.21.15.47:0/967185212 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff72801a070 con 0x7ff7300c6880 2024-04-24T22:44:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.708+0000 7ff727400640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff71006be60 0x7ff71006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:39.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.708+0000 7ff727400640 1 -- 172.21.15.47:0/967185212 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff72808a200 con 0x7ff7300c6880 2024-04-24T22:44:39.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.708+0000 7ff72f400640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff71006be60 0x7ff71006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:39.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.709+0000 7ff72f400640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff71006be60 0x7ff71006e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff72000c090 tx=0x7ff72002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:39.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.712+0000 7ff727400640 1 -- 172.21.15.47:0/967185212 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff728058470 con 0x7ff7300c6880 2024-04-24T22:44:39.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.954+0000 7ff735200640 1 -- 172.21.15.47:0/967185212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff73015d290 con 0x7ff7300c6880 2024-04-24T22:44:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.964+0000 7ff727400640 1 -- 172.21.15.47:0/967185212 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7ff73015d290 con 0x7ff7300c6880 2024-04-24T22:44:39.967 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T22:44:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 -- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff71006be60 msgr2=0x7ff71006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff71006be60 0x7ff71006e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff72000c090 tx=0x7ff72002e000 comp rx=0 tx=0).stop 2024-04-24T22:44:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 -- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7300c6880 msgr2=0x7ff73014cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7300c6880 0x7ff73014cc50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff72800c410 tx=0x7ff72800c8d0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 -- 172.21.15.47:0/967185212 shutdown_connections 2024-04-24T22:44:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7ff71006be60 0x7ff71006e320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7300c6880 0x7ff73014cc50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7300c2850 0x7ff73014c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 --2- 172.21.15.47:0/967185212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff73006e1b0 0x7ff730144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 -- 172.21.15.47:0/967185212 >> 172.21.15.47:0/967185212 conn(0x7ff7300cb900 msgr2=0x7ff730150ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 -- 172.21.15.47:0/967185212 shutdown_connections 2024-04-24T22:44:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:39.967+0000 7ff735200640 1 -- 172.21.15.47:0/967185212 wait complete. 2024-04-24T22:44:39.990 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T22:44:40.003 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-04-24T22:44:40.003 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-04-24T22:44:40.073 INFO:tasks.ceph.mds.e.smithi047.stdout:starting mds.e at 2024-04-24T22:44:40.089 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-24T22:44:40.094 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-04-24T22:44:40.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.188+0000 7fe05d600640 1 Processor -- start 2024-04-24T22:44:40.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.188+0000 7fe05d600640 1 -- start start 2024-04-24T22:44:40.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.189+0000 7fe05d600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0580c8fa0 0x7fe058163dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:40.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.189+0000 7fe05d600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0580c7bb0 con 0x7fe0580c7750 2024-04-24T22:44:40.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.189+0000 7fe05d600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0580c7e50 con 0x7fe058152960 2024-04-24T22:44:40.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.189+0000 7fe05d600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe058158f20 con 0x7fe0580c8fa0 2024-04-24T22:44:40.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.190+0000 7fe056a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe058152960 0x7fe05815ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.47:43260/0 (socket says 172.21.15.47:43260) 2024-04-24T22:44:40.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.190+0000 7fe056a00640 1 -- 172.21.15.47:0/2308918691 learned_addr learned my addr 172.21.15.47:0/2308918691 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:40.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.190+0000 7fe057400640 1 --2- 172.21.15.47:0/2308918691 >> v2:172.21.15.47:3300/0 conn(0x7fe0580c8fa0 0x7fe058163dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:40.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.190+0000 7fe056000640 1 -- 172.21.15.47:0/2308918691 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 541922911 0 0) 0x7fe0580c7e50 con 0x7fe058152960 2024-04-24T22:44:40.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.190+0000 7fe056000640 1 -- 172.21.15.47:0/2308918691 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe03c003520 con 0x7fe058152960 2024-04-24T22:44:40.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.190+0000 7fe057400640 1 -- 172.21.15.47:0/2308918691 >> v1:172.21.15.113:6789/0 conn(0x7fe058152960 legacy=0x7fe05815ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:40.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.190+0000 7fe057400640 1 -- 172.21.15.47:0/2308918691 >> v1:172.21.15.47:6789/0 conn(0x7fe0580c7750 legacy=0x7fe058072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:40.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.190+0000 7fe057400640 1 -- 172.21.15.47:0/2308918691 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0581585f0 con 0x7fe0580c8fa0 2024-04-24T22:44:40.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.191+0000 7fe057400640 1 --2- 172.21.15.47:0/2308918691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0580c8fa0 0x7fe058163dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe050009d90 tx=0x7fe05002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7fddb21f6b8fc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:40.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.192+0000 7fe056000640 1 -- 172.21.15.47:0/2308918691 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050037070 con 0x7fe0580c8fa0 2024-04-24T22:44:40.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.192+0000 7fe056000640 1 -- 172.21.15.47:0/2308918691 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe050007740 con 0x7fe0580c8fa0 2024-04-24T22:44:40.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.192+0000 7fe056000640 1 -- 172.21.15.47:0/2308918691 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050007af0 con 0x7fe0580c8fa0 2024-04-24T22:44:40.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.192+0000 7fe05d600640 1 -- 172.21.15.47:0/2308918691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0580c8fa0 msgr2=0x7fe058163dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.192+0000 7fe05d600640 1 --2- 172.21.15.47:0/2308918691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0580c8fa0 0x7fe058163dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe050009d90 tx=0x7fe05002a780 comp rx=0 tx=0).stop 2024-04-24T22:44:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.192+0000 7fe05d600640 1 -- 172.21.15.47:0/2308918691 shutdown_connections 2024-04-24T22:44:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.192+0000 7fe05d600640 1 --2- 172.21.15.47:0/2308918691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0580c8fa0 0x7fe058163dc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.192+0000 7fe05d600640 1 -- 172.21.15.47:0/2308918691 >> 172.21.15.47:0/2308918691 conn(0x7fe05806e580 msgr2=0x7fe05806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.192+0000 7fe05d600640 1 -- 172.21.15.47:0/2308918691 shutdown_connections 2024-04-24T22:44:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.192+0000 7fe05d600640 1 -- 172.21.15.47:0/2308918691 wait complete. 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.194+0000 7fe05d600640 1 Processor -- start 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.194+0000 7fe05d600640 1 -- start start 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe05d600640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580c7750 0x7fe0580c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe05d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0580c8fa0 0x7fe0580c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe05d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe058152960 0x7fe058146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe05d600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe0580c8130 con 0x7fe058152960 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe05d600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581593b0 con 0x7fe0580c8fa0 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe05d600640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe058158bb0 con 0x7fe0580c7750 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe056a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580c7750 0x7fe0580c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe04fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0580c8fa0 0x7fe0580c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe056a00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580c7750 0x7fe0580c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.47:43662/0 (socket says 172.21.15.47:43662) 2024-04-24T22:44:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe056a00640 1 -- 172.21.15.47:0/1887930502 learned_addr learned my addr 172.21.15.47:0/1887930502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.195+0000 7fe057400640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe058152960 0x7fe058146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.196+0000 7fe056a00640 1 -- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0580c8fa0 msgr2=0x7fe0580c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.196+0000 7fe056a00640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0580c8fa0 0x7fe0580c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.196+0000 7fe056a00640 1 -- 172.21.15.47:0/1887930502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe058152960 msgr2=0x7fe058146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.196+0000 7fe056a00640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe058152960 0x7fe058146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.196+0000 7fe056a00640 1 -- 172.21.15.47:0/1887930502 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05814ad40 con 0x7fe0580c7750 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.196+0000 7fe056a00640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580c7750 0x7fe0580c6460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe04800c440 tx=0x7fe04800c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.196+0000 7fe04ea00640 1 -- 172.21.15.47:0/1887930502 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048017070 con 0x7fe0580c7750 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.196+0000 7fe04ea00640 1 -- 172.21.15.47:0/1887930502 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe048005ce0 con 0x7fe0580c7750 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.196+0000 7fe04ea00640 1 -- 172.21.15.47:0/1887930502 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0480072d0 con 0x7fe0580c7750 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.196+0000 7fe05d600640 1 -- 172.21.15.47:0/1887930502 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05814b020 con 0x7fe0580c7750 2024-04-24T22:44:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.197+0000 7fe05d600640 1 -- 172.21.15.47:0/1887930502 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05814b5e0 con 0x7fe0580c7750 2024-04-24T22:44:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.198+0000 7fe05d600640 1 -- 172.21.15.47:0/1887930502 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe058073b90 con 0x7fe0580c7750 2024-04-24T22:44:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.199+0000 7fe04ea00640 1 -- 172.21.15.47:0/1887930502 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe04801a070 con 0x7fe0580c7750 2024-04-24T22:44:40.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.199+0000 7fe04ea00640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe03406be30 0x7fe03406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.200+0000 7fe04ea00640 1 -- 172.21.15.47:0/1887930502 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6255+0+0 (secure 0 0 0) 0x7fe04808a4d0 con 0x7fe0580c7750 2024-04-24T22:44:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.200+0000 7fe04fe00640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe03406be30 0x7fe03406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.200+0000 7fe04fe00640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe03406be30 0x7fe03406e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe0581f13b0 tx=0x7fe040009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.204+0000 7fe04ea00640 1 -- 172.21.15.47:0/1887930502 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe0480585a0 con 0x7fe0580c7750 2024-04-24T22:44:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.438+0000 7fe05d600640 1 -- 172.21.15.47:0/1887930502 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe05806c200 con 0x7fe0580c7750 2024-04-24T22:44:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.964+0000 7fe04ea00640 1 -- 172.21.15.47:0/1887930502 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fe04805c8a0 con 0x7fe0580c7750 2024-04-24T22:44:40.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.969+0000 7fe05d600640 1 -- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe03406be30 msgr2=0x7fe03406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.969+0000 7fe05d600640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe03406be30 0x7fe03406e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe0581f13b0 tx=0x7fe040009040 comp rx=0 tx=0).stop 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.969+0000 7fe05d600640 1 -- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580c7750 msgr2=0x7fe0580c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.969+0000 7fe05d600640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580c7750 0x7fe0580c6460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe04800c440 tx=0x7fe04800c900 comp rx=0 tx=0).stop 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.970+0000 7fe05d600640 1 -- 172.21.15.47:0/1887930502 shutdown_connections 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.970+0000 7fe05d600640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7fe03406be30 0x7fe03406e2f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.970+0000 7fe05d600640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe058152960 0x7fe058146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.970+0000 7fe05d600640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0580c8fa0 0x7fe0580c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.970+0000 7fe05d600640 1 --2- 172.21.15.47:0/1887930502 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580c7750 0x7fe0580c6460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.970+0000 7fe05d600640 1 -- 172.21.15.47:0/1887930502 >> 172.21.15.47:0/1887930502 conn(0x7fe05806e580 msgr2=0x7fe05806e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.970+0000 7fe05d600640 1 -- 172.21.15.47:0/1887930502 shutdown_connections 2024-04-24T22:44:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:40.970+0000 7fe05d600640 1 -- 172.21.15.47:0/1887930502 wait complete. 2024-04-24T22:44:40.994 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T22:44:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.190+0000 7f7b15c00640 1 Processor -- start 2024-04-24T22:44:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.190+0000 7f7b15c00640 1 -- start start 2024-04-24T22:44:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.192+0000 7f7b15c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b10152600 0x7f7b1015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.192+0000 7f7b15c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b10158d60 0x7f7b10072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.192+0000 7f7b15c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b100c57e0 con 0x7f7b100c9270 2024-04-24T22:44:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.192+0000 7f7b15c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7b100c6010 con 0x7f7b10152600 2024-04-24T22:44:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.192+0000 7f7b15c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7b101529e0 con 0x7f7b10158d60 2024-04-24T22:44:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.192+0000 7f7b0ea00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b10152600 0x7f7b1015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.193+0000 7f7b0ea00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b10152600 0x7f7b1015c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42288/0 (socket says 172.21.15.47:42288) 2024-04-24T22:44:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.193+0000 7f7b0ea00640 1 -- 172.21.15.47:0/3744979428 learned_addr learned my addr 172.21.15.47:0/3744979428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.193+0000 7f7b0f400640 1 --2- 172.21.15.47:0/3744979428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b10158d60 0x7f7b10072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.193+0000 7f7b0ea00640 1 -- 172.21.15.47:0/3744979428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b10158d60 msgr2=0x7f7b10072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T22:44:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.193+0000 7f7b0ea00640 1 --2- 172.21.15.47:0/3744979428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b10158d60 0x7f7b10072bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.193+0000 7f7b0ea00640 1 -- 172.21.15.47:0/3744979428 >> v1:172.21.15.47:6789/0 conn(0x7f7b100c9270 legacy=0x7f7b100c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.193+0000 7f7b0ea00640 1 -- 172.21.15.47:0/3744979428 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b100c5a50 con 0x7f7b10152600 2024-04-24T22:44:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.194+0000 7f7b0ea00640 1 --2- 172.21.15.47:0/3744979428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10152600 0x7f7b1015c4e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009850 tx=0x7f7b0002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96f825589f02834 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.195+0000 7f7b0d600640 1 -- 172.21.15.47:0/3744979428 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00004030 con 0x7f7b10152600 2024-04-24T22:44:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.195+0000 7f7b0d600640 1 -- 172.21.15.47:0/3744979428 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7b000041d0 con 0x7f7b10152600 2024-04-24T22:44:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.195+0000 7f7b0d600640 1 -- 172.21.15.47:0/3744979428 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00003610 con 0x7f7b10152600 2024-04-24T22:44:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.195+0000 7f7b15c00640 1 -- 172.21.15.47:0/3744979428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10152600 msgr2=0x7f7b1015c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.195+0000 7f7b15c00640 1 --2- 172.21.15.47:0/3744979428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10152600 0x7f7b1015c4e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009850 tx=0x7f7b0002a1a0 comp rx=0 tx=0).stop 2024-04-24T22:44:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.195+0000 7f7b15c00640 1 -- 172.21.15.47:0/3744979428 shutdown_connections 2024-04-24T22:44:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.195+0000 7f7b15c00640 1 --2- 172.21.15.47:0/3744979428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b10158d60 0x7f7b10072bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.195+0000 7f7b15c00640 1 --2- 172.21.15.47:0/3744979428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10152600 0x7f7b1015c4e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.195+0000 7f7b15c00640 1 -- 172.21.15.47:0/3744979428 >> 172.21.15.47:0/3744979428 conn(0x7f7b100c86a0 msgr2=0x7f7b100c8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.195+0000 7f7b15c00640 1 -- 172.21.15.47:0/3744979428 shutdown_connections 2024-04-24T22:44:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.196+0000 7f7b15c00640 1 -- 172.21.15.47:0/3744979428 wait complete. 2024-04-24T22:44:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.197+0000 7f7b15c00640 1 Processor -- start 2024-04-24T22:44:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.197+0000 7f7b15c00640 1 -- start start 2024-04-24T22:44:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.198+0000 7f7b15c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b100c9270 0x7f7b101f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.198+0000 7f7b15c00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b10152600 0x7f7b1015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.198+0000 7f7b15c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10158d60 0x7f7b101e2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.198+0000 7f7b15c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1015bce0 con 0x7f7b10158d60 2024-04-24T22:44:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.198+0000 7f7b15c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1015ba00 con 0x7f7b100c9270 2024-04-24T22:44:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.198+0000 7f7b15c00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1015bfc0 con 0x7f7b10152600 2024-04-24T22:44:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10158d60 0x7f7b101e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10158d60 0x7f7b101e2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42300/0 (socket says 172.21.15.47:42300) 2024-04-24T22:44:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0f400640 1 -- 172.21.15.47:0/2773679200 learned_addr learned my addr 172.21.15.47:0/2773679200 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0ea00640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b100c9270 0x7f7b101f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0e000640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b10152600 0x7f7b1015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0f400640 1 -- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b10152600 msgr2=0x7f7b1015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0f400640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b10152600 0x7f7b1015a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0f400640 1 -- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b100c9270 msgr2=0x7f7b101f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0f400640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b100c9270 0x7f7b101f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0f400640 1 -- 172.21.15.47:0/2773679200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1015aa80 con 0x7f7b10158d60 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.199+0000 7f7b0f400640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10158d60 0x7f7b101e2860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009dd0 tx=0x7f7b0800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.200+0000 7f7b06a00640 1 -- 172.21.15.47:0/2773679200 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08018020 con 0x7f7b10158d60 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.200+0000 7f7b06a00640 1 -- 172.21.15.47:0/2773679200 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7b08014480 con 0x7f7b10158d60 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.200+0000 7f7b06a00640 1 -- 172.21.15.47:0/2773679200 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08014890 con 0x7f7b10158d60 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.200+0000 7f7b15c00640 1 -- 172.21.15.47:0/2773679200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1014d420 con 0x7f7b10158d60 2024-04-24T22:44:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.200+0000 7f7b15c00640 1 -- 172.21.15.47:0/2773679200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b101594a0 con 0x7f7b10158d60 2024-04-24T22:44:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.201+0000 7f7b15c00640 1 -- 172.21.15.47:0/2773679200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b10073c00 con 0x7f7b10158d60 2024-04-24T22:44:41.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.202+0000 7f7b06a00640 1 -- 172.21.15.47:0/2773679200 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b08005d10 con 0x7f7b10158d60 2024-04-24T22:44:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.203+0000 7f7b06a00640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7aec06be30 0x7f7aec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.203+0000 7f7b06a00640 1 -- 172.21.15.47:0/2773679200 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6255+0+0 (secure 0 0 0) 0x7f7b0808bf70 con 0x7f7b10158d60 2024-04-24T22:44:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.204+0000 7f7b0ea00640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7aec06be30 0x7f7aec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.205+0000 7f7b0ea00640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7aec06be30 0x7f7aec06e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b00033000 tx=0x7f7b0002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:41.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.208+0000 7f7b06a00640 1 -- 172.21.15.47:0/2773679200 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7b08022030 con 0x7f7b10158d60 2024-04-24T22:44:41.292 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:44:41.299 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:44:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:41.416+0000 7f7b15c00640 1 -- 172.21.15.47:0/2773679200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7b101501d0 con 0x7f7b10158d60 2024-04-24T22:44:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.044+0000 7f7b06a00640 1 -- 172.21.15.47:0/2773679200 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7f7b08010040 con 0x7f7b10158d60 2024-04-24T22:44:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-04-24T22:44:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.048+0000 7f7b15c00640 1 -- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7aec06be30 msgr2=0x7f7aec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.048+0000 7f7b15c00640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7aec06be30 0x7f7aec06e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b00033000 tx=0x7f7b0002e000 comp rx=0 tx=0).stop 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.048+0000 7f7b15c00640 1 -- 172.21.15.47:0/2773679200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10158d60 msgr2=0x7f7b101e2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.048+0000 7f7b15c00640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10158d60 0x7f7b101e2860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009dd0 tx=0x7f7b0800c490 comp rx=0 tx=0).stop 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.049+0000 7f7b15c00640 1 -- 172.21.15.47:0/2773679200 shutdown_connections 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.049+0000 7f7b15c00640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f7aec06be30 0x7f7aec06e2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.049+0000 7f7b15c00640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b10158d60 0x7f7b101e2860 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.049+0000 7f7b15c00640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b10152600 0x7f7b1015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.049+0000 7f7b15c00640 1 --2- 172.21.15.47:0/2773679200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b100c9270 0x7f7b101f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.049+0000 7f7b15c00640 1 -- 172.21.15.47:0/2773679200 >> 172.21.15.47:0/2773679200 conn(0x7f7b100c86a0 msgr2=0x7f7b1014e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.049+0000 7f7b15c00640 1 -- 172.21.15.47:0/2773679200 shutdown_connections 2024-04-24T22:44:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.049+0000 7f7b15c00640 1 -- 172.21.15.47:0/2773679200 wait complete. 2024-04-24T22:44:42.080 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T22:44:42.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.341+0000 7f8be6400640 1 Processor -- start 2024-04-24T22:44:42.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.341+0000 7f8be6400640 1 -- start start 2024-04-24T22:44:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.343+0000 7f8be6400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 0x7f8be00c6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.343+0000 7f8be6400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be00c6cd0 con 0x7f8be0158d30 2024-04-24T22:44:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.343+0000 7f8be6400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be015e840 con 0x7f8be00c1cb0 2024-04-24T22:44:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.343+0000 7f8be6400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8be00c7610 con 0x7f8be0158950 2024-04-24T22:44:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.344+0000 7f8bdfe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8be0158d30 0x7f8be00c15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51614/0 (socket says 172.21.15.47:51614) 2024-04-24T22:44:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.344+0000 7f8bdfe00640 1 -- 172.21.15.47:0/3191248525 learned_addr learned my addr 172.21.15.47:0/3191248525 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-24T22:44:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.344+0000 7f8be5000640 1 --2- 172.21.15.47:0/3191248525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 0x7f8be00c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.344+0000 7f8be5000640 1 -- 172.21.15.47:0/3191248525 >> v1:172.21.15.113:6789/0 conn(0x7f8be00c1cb0 legacy=0x7f8be015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.344+0000 7f8be5000640 1 -- 172.21.15.47:0/3191248525 >> v1:172.21.15.47:6789/0 conn(0x7f8be0158d30 legacy=0x7f8be00c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.344+0000 7f8be5000640 1 -- 172.21.15.47:0/3191248525 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be00c7860 con 0x7f8be0158950 2024-04-24T22:44:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.344+0000 7f8bdf400640 1 -- 172.21.15.47:0/3191248525 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 192229964 0 0) 0x7f8be00c6cd0 con 0x7f8be0158d30 2024-04-24T22:44:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.345+0000 7f8be5000640 1 --2- 172.21.15.47:0/3191248525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 0x7f8be00c6790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009dc0 tx=0x7f8bd002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b70d92e6dc6f0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.346+0000 7f8bdf400640 1 -- 172.21.15.47:0/3191248525 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0007740 con 0x7f8be0158950 2024-04-24T22:44:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.346+0000 7f8bdf400640 1 -- 172.21.15.47:0/3191248525 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8bd00078e0 con 0x7f8be0158950 2024-04-24T22:44:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.346+0000 7f8bdf400640 1 -- 172.21.15.47:0/3191248525 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0007cd0 con 0x7f8be0158950 2024-04-24T22:44:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.346+0000 7f8be6400640 1 -- 172.21.15.47:0/3191248525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 msgr2=0x7f8be00c6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.346+0000 7f8be6400640 1 --2- 172.21.15.47:0/3191248525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 0x7f8be00c6790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009dc0 tx=0x7f8bd002a6d0 comp rx=0 tx=0).stop 2024-04-24T22:44:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.346+0000 7f8be6400640 1 -- 172.21.15.47:0/3191248525 shutdown_connections 2024-04-24T22:44:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.346+0000 7f8be6400640 1 --2- 172.21.15.47:0/3191248525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 0x7f8be00c6790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.346+0000 7f8be6400640 1 -- 172.21.15.47:0/3191248525 >> 172.21.15.47:0/3191248525 conn(0x7f8be00cbc30 msgr2=0x7f8be00cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.346+0000 7f8be6400640 1 -- 172.21.15.47:0/3191248525 shutdown_connections 2024-04-24T22:44:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.346+0000 7f8be6400640 1 -- 172.21.15.47:0/3191248525 wait complete. 2024-04-24T22:44:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.348+0000 7f8be6400640 1 Processor -- start 2024-04-24T22:44:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.348+0000 7f8be6400640 1 -- start start 2024-04-24T22:44:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.349+0000 7f8be6400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 0x7f8be01543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.349+0000 7f8be6400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8be0158d30 0x7f8be01f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.349+0000 7f8be6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be01579f0 0x7f8be01f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.349+0000 7f8be6400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00c8100 con 0x7f8be0158d30 2024-04-24T22:44:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.349+0000 7f8be6400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00c7e20 con 0x7f8be01579f0 2024-04-24T22:44:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.349+0000 7f8be6400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8be00c83e0 con 0x7f8be0158950 2024-04-24T22:44:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8be5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be01579f0 0x7f8be01f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8be5000640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 0x7f8be01543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8be5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be01579f0 0x7f8be01f4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.47:57848/0 (socket says 172.21.15.47:57848) 2024-04-24T22:44:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8be5a00640 1 -- 172.21.15.47:0/818183390 learned_addr learned my addr 172.21.15.47:0/818183390 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-24T22:44:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8bdfe00640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8be0158d30 0x7f8be01f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8be5a00640 1 -- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 msgr2=0x7f8be01543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8be5a00640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 0x7f8be01543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8be5a00640 1 -- 172.21.15.47:0/818183390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8be0158d30 msgr2=0x7f8be01f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8be5a00640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8be0158d30 0x7f8be01f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8be5a00640 1 -- 172.21.15.47:0/818183390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be00c5370 con 0x7f8be01579f0 2024-04-24T22:44:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.350+0000 7f8be5000640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 0x7f8be01543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T22:44:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.351+0000 7f8be5a00640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be01579f0 0x7f8be01f4c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800c3a0 tx=0x7f8bd800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.351+0000 7f8bdcc00640 1 -- 172.21.15.47:0/818183390 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8017070 con 0x7f8be01579f0 2024-04-24T22:44:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.351+0000 7f8bdcc00640 1 -- 172.21.15.47:0/818183390 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8bd80052d0 con 0x7f8be01579f0 2024-04-24T22:44:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.351+0000 7f8bdcc00640 1 -- 172.21.15.47:0/818183390 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8003850 con 0x7f8be01579f0 2024-04-24T22:44:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.351+0000 7f8be6400640 1 -- 172.21.15.47:0/818183390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be00c55f0 con 0x7f8be01579f0 2024-04-24T22:44:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.351+0000 7f8be6400640 1 -- 172.21.15.47:0/818183390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be0072860 con 0x7f8be01579f0 2024-04-24T22:44:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.352+0000 7f8be6400640 1 -- 172.21.15.47:0/818183390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be0069810 con 0x7f8be01579f0 2024-04-24T22:44:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.353+0000 7f8bdcc00640 1 -- 172.21.15.47:0/818183390 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bd801a070 con 0x7f8be01579f0 2024-04-24T22:44:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.354+0000 7f8bdcc00640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8bbc06be30 0x7f8bbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T22:44:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.354+0000 7f8bdcc00640 1 -- 172.21.15.47:0/818183390 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5817+0+0 (secure 0 0 0) 0x7f8bd808a210 con 0x7f8be01579f0 2024-04-24T22:44:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.354+0000 7f8be5000640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8bbc06be30 0x7f8bbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T22:44:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.355+0000 7f8be5000640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8bbc06be30 0x7f8bbc06e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0034000 tx=0x7f8bd002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T22:44:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.359+0000 7f8bdcc00640 1 -- 172.21.15.47:0/818183390 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8be00c5370 con 0x7f8be01579f0 2024-04-24T22:44:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:42.563+0000 7f8be6400640 1 -- 172.21.15.47:0/818183390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8be0073b90 con 0x7f8be01579f0 2024-04-24T22:44:43.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.063+0000 7f8bdcc00640 1 -- 172.21.15.47:0/818183390 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f8bd8058520 con 0x7f8be01579f0 2024-04-24T22:44:43.067 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-04-24T22:44:43.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.069+0000 7f8be6400640 1 -- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8bbc06be30 msgr2=0x7f8bbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:43.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.069+0000 7f8be6400640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8bbc06be30 0x7f8bbc06e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0034000 tx=0x7f8bd002f000 comp rx=0 tx=0).stop 2024-04-24T22:44:43.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.069+0000 7f8be6400640 1 -- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be01579f0 msgr2=0x7f8be01f4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T22:44:43.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.069+0000 7f8be6400640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be01579f0 0x7f8be01f4c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800c3a0 tx=0x7f8bd800c860 comp rx=0 tx=0).stop 2024-04-24T22:44:43.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.069+0000 7f8be6400640 1 -- 172.21.15.47:0/818183390 shutdown_connections 2024-04-24T22:44:43.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.069+0000 7f8be6400640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:6832/2101666090,v1:172.21.15.113:6833/2101666090] conn(0x7f8bbc06be30 0x7f8bbc06e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:43.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.069+0000 7f8be6400640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8be01579f0 0x7f8be01f4c50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:43.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.069+0000 7f8be6400640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8be0158d30 0x7f8be01f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:43.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.069+0000 7f8be6400640 1 --2- 172.21.15.47:0/818183390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8be0158950 0x7f8be01543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T22:44:43.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.069+0000 7f8be6400640 1 -- 172.21.15.47:0/818183390 >> 172.21.15.47:0/818183390 conn(0x7f8be00cbc30 msgr2=0x7f8be015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T22:44:43.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.074+0000 7f8be6400640 1 -- 172.21.15.47:0/818183390 shutdown_connections 2024-04-24T22:44:43.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-24T22:44:43.075+0000 7f8be6400640 1 -- 172.21.15.47:0/818183390 wait complete. 2024-04-24T22:44:43.106 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-24T22:44:43.106 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T22:44:43.106 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.128 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T22:44:43.128 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T22:44:43.128 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.186 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T22:44:43.186 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-24T22:44:43.186 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.246 INFO:tasks.ceph.mds.e:Stopped 2024-04-24T22:44:43.246 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T22:44:43.246 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.258 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T22:44:43.258 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T22:44:43.258 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.309 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T22:44:43.310 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-24T22:44:43.310 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.361 INFO:tasks.ceph.mds.f:Stopped 2024-04-24T22:44:43.361 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-24T22:44:43.361 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-24T22:44:43.361 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.455 INFO:tasks.ceph.osd.0:Stopped 2024-04-24T22:44:43.455 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-24T22:44:43.455 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.530 INFO:tasks.ceph.osd.1:Stopped 2024-04-24T22:44:43.531 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-24T22:44:43.531 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.592 INFO:tasks.ceph.osd.2:Stopped 2024-04-24T22:44:43.592 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-24T22:44:43.592 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.678 INFO:tasks.ceph.osd.3:Stopped 2024-04-24T22:44:43.678 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-24T22:44:43.678 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.713 INFO:tasks.ceph.osd.4:Stopped 2024-04-24T22:44:43.713 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-24T22:44:43.713 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.783 INFO:tasks.ceph.osd.5:Stopped 2024-04-24T22:44:43.784 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-24T22:44:43.784 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.861 INFO:tasks.ceph.osd.6:Stopped 2024-04-24T22:44:43.862 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-24T22:44:43.862 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:43.939 INFO:tasks.ceph.osd.7:Stopped 2024-04-24T22:44:43.939 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-24T22:44:43.939 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-24T22:44:43.939 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:47.805 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-24T22:44:50.041 INFO:tasks.ceph.mgr.y:Stopped 2024-04-24T22:44:50.042 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-24T22:44:50.042 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:50.072 INFO:tasks.ceph.mgr.x:Stopped 2024-04-24T22:44:50.072 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-24T22:44:50.072 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-24T22:44:50.072 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:50.119 INFO:tasks.ceph.mon.a:Stopped 2024-04-24T22:44:50.120 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-24T22:44:50.120 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:50.133 INFO:tasks.ceph.mon.b:Stopped 2024-04-24T22:44:50.133 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-24T22:44:50.133 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T22:44:50.188 INFO:tasks.ceph.mon.c:Stopped 2024-04-24T22:44:50.188 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-24T22:44:50.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-04-24T22:44:50.313 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:44:50.314 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-24T22:44:50.498 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:44:50.498 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-24T22:44:50.684 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:44:50.685 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-24T22:44:50.823 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:44:50.823 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-24T22:44:50.988 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:44:50.989 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-24T22:44:51.202 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:44:51.202 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-24T22:44:51.282 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:44:51.282 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-24T22:44:51.340 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:44:51.340 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-24T22:44:51.421 INFO:tasks.ceph:Archiving mon data... 2024-04-24T22:44:51.422 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135/data/mon.a.tgz 2024-04-24T22:44:51.423 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-24T22:44:51.484 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.iZK767wjGM 2024-04-24T22:44:51.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.iZK767wjGM 2024-04-24T22:44:51.654 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.iZK767wjGM 2024-04-24T22:44:51.798 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.iZK767wjGM is 337KB 2024-04-24T22:44:51.914 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.iZK767wjGM 2024-04-24T22:44:51.973 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135/data/mon.b.tgz 2024-04-24T22:44:51.973 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-04-24T22:44:51.992 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.SJhK6Wovof 2024-04-24T22:44:51.993 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.SJhK6Wovof 2024-04-24T22:44:52.131 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.SJhK6Wovof 2024-04-24T22:44:52.214 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.SJhK6Wovof is 357KB 2024-04-24T22:44:52.310 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.SJhK6Wovof 2024-04-24T22:44:52.325 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135/data/mon.c.tgz 2024-04-24T22:44:52.325 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-04-24T22:44:52.379 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.0fcw5bBcne 2024-04-24T22:44:52.380 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.0fcw5bBcne 2024-04-24T22:44:52.506 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.0fcw5bBcne 2024-04-24T22:44:52.587 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.0fcw5bBcne is 356KB 2024-04-24T22:44:52.683 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.0fcw5bBcne 2024-04-24T22:44:52.698 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-24T22:44:52.698 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T22:44:52.706 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T22:44:52.805 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-24T22:44:52.848 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-24T22:44:52.862 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-24T22:44:52.862 INFO:tasks.ceph:Archiving crash dumps... 2024-04-24T22:44:52.864 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135/remote/smithi047/crash 2024-04-24T22:44:52.865 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T22:44:52.952 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135/remote/smithi113/crash 2024-04-24T22:44:52.953 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T22:44:52.976 INFO:tasks.ceph:Shutting down logrotate 2024-04-24T22:44:52.976 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:44:52.983 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T22:44:53.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-24T22:44:53.183 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-24T22:44:53.206 INFO:tasks.ceph:Compressing logs... 2024-04-24T22:44:53.206 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T22:44:53.214 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T22:44:53.276 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34936.log 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34936.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.7.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34936.log.gz 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34816.log 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-24T22:44:53.277 INFO:teuthology.orchestra.run.smithi113.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34816.log: gzip -5 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr: --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7%/var/log/ceph/ceph-osd.4.log: -- replaced with /var/log/ceph/ceph-client.admin.34816.log.gz 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35016.log 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34765.log 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35016.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35016.log.gz 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34725.log 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34765.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34765.log.gz 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34896.log 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34725.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34725.log.gz 2024-04-24T22:44:53.278 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34856.log 2024-04-24T22:44:53.282 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34896.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34896.log.gz 2024-04-24T22:44:53.282 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34976.log 2024-04-24T22:44:53.282 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34856.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34856.log.gz 2024-04-24T22:44:53.282 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-24T22:44:53.282 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34976.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34976.log.gz 2024-04-24T22:44:53.282 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-24T22:44:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46095.log 2024-04-24T22:44:53.292 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44751.log 2024-04-24T22:44:53.293 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43617.log 2024-04-24T22:44:53.293 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46095.log.gz 2024-04-24T22:44:53.294 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44751.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44751.log.gz 2024-04-24T22:44:53.294 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38949.log 2024-04-24T22:44:53.295 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42248.log 2024-04-24T22:44:53.296 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43617.log.gz 2024-04-24T22:44:53.296 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41974.log 2024-04-24T22:44:53.297 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38949.log.gz 2024-04-24T22:44:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42248.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42248.log.gz 2024-04-24T22:44:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39719.log 2024-04-24T22:44:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41114.log 2024-04-24T22:44:53.299 INFO:teuthology.orchestra.run.smithi047.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41974.log.gz 2024-04-24T22:44:53.299 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40260.log 2024-04-24T22:44:53.300 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39719.log.gz 2024-04-24T22:44:53.301 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41114.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41114.log.gz 2024-04-24T22:44:53.301 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42851.log 2024-04-24T22:44:53.302 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-24T22:44:53.302 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44974.log 2024-04-24T22:44:53.302 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42851.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40260.log.gz 2024-04-24T22:44:53.303 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.42766.log -- replaced with /var/log/ceph/ceph-client.admin.42851.log.gz 2024-04-24T22:44:53.303 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-24T22:44:53.304 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45998.log 2024-04-24T22:44:53.304 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44974.log.gz 2024-04-24T22:44:53.305 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39331.log 2024-04-24T22:44:53.305 INFO:teuthology.orchestra.run.smithi047.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42766.log.gz 2024-04-24T22:44:53.306 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45998.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45998.log.gz 2024-04-24T22:44:53.306 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37618.log 2024-04-24T22:44:53.307 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38100.log 2024-04-24T22:44:53.307 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39331.log.gz 2024-04-24T22:44:53.308 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45202.log 2024-04-24T22:44:53.309 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37618.log.gz 2024-04-24T22:44:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38100.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38100.log.gz 2024-04-24T22:44:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38606.log 2024-04-24T22:44:53.311 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42981.log 2024-04-24T22:44:53.311 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45202.log.gz 2024-04-24T22:44:53.311 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39891.log 2024-04-24T22:44:53.312 INFO:teuthology.orchestra.run.smithi113.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-24T22:44:53.313 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38606.log.gz 2024-04-24T22:44:53.313 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40128.log 2024-04-24T22:44:53.313 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42981.log.gz 2024-04-24T22:44:53.313 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39891.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T22:44:53.314 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39891.log.gz 2024-04-24T22:44:53.314 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43702.log 2024-04-24T22:44:53.315 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40128.log.gz 2024-04-24T22:44:53.316 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45454.log 2024-04-24T22:44:53.317 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40062.log 2024-04-24T22:44:53.317 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43702.log.gz 2024-04-24T22:44:53.317 INFO:teuthology.orchestra.run.smithi047.stderr: 85.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T22:44:53.318 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43828.log 2024-04-24T22:44:53.318 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45454.log.gz 2024-04-24T22:44:53.318 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41421.log 2024-04-24T22:44:53.319 INFO:teuthology.orchestra.run.smithi047.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40062.log.gz 2024-04-24T22:44:53.320 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43828.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-24T22:44:53.320 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43828.log.gz 2024-04-24T22:44:53.321 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37422.log 2024-04-24T22:44:53.321 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41421.log.gz 2024-04-24T22:44:53.322 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44014.log 2024-04-24T22:44:53.323 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46241.log 2024-04-24T22:44:53.323 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37422.log.gz 2024-04-24T22:44:53.324 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45538.log 2024-04-24T22:44:53.324 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46241.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44014.log.gz 2024-04-24T22:44:53.325 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39551.log 2024-04-24T22:44:53.325 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46241.log.gz 2024-04-24T22:44:53.326 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-04-24T22:44:53.326 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45538.log.gz 2024-04-24T22:44:53.327 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39551.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.39551.log.gz 2024-04-24T22:44:53.328 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43296.log 2024-04-24T22:44:53.329 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41298.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-04-24T22:44:53.329 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43930.log 2024-04-24T22:44:53.330 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39849.log 2024-04-24T22:44:53.330 INFO:teuthology.orchestra.run.smithi047.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.43296.log.gz 2024-04-24T22:44:53.331 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46181.log 2024-04-24T22:44:53.331 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43930.log.gz 2024-04-24T22:44:53.332 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39849.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T22:44:53.332 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39849.log.gz 2024-04-24T22:44:53.333 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43870.log 2024-04-24T22:44:53.333 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46181.log.gz 2024-04-24T22:44:53.334 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41932.log 2024-04-24T22:44:53.335 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43870.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T22:44:53.335 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39425.log 2024-04-24T22:44:53.335 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43870.log.gz 2024-04-24T22:44:53.336 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45370.log 2024-04-24T22:44:53.336 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41932.log.gz 2024-04-24T22:44:53.337 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43445.log 2024-04-24T22:44:53.337 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39425.log.gz 2024-04-24T22:44:53.338 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42500.log 2024-04-24T22:44:53.339 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45370.log.gz 2024-04-24T22:44:53.339 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44098.log 2024-04-24T22:44:53.339 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43445.log.gz 2024-04-24T22:44:53.340 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45748.log 2024-04-24T22:44:53.340 INFO:teuthology.orchestra.run.smithi047.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42500.log.gz 2024-04-24T22:44:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39281.log 2024-04-24T22:44:53.341 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44098.log.gz 2024-04-24T22:44:53.342 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45748.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45748.log.gz 2024-04-24T22:44:53.343 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45328.log 2024-04-24T22:44:53.343 INFO:teuthology.orchestra.run.smithi113.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-24T22:44:53.344 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38612.log 2024-04-24T22:44:53.344 INFO:teuthology.orchestra.run.smithi047.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39281.log.gz 2024-04-24T22:44:53.344 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45328.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45328.log.gz 2024-04-24T22:44:53.345 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38940.log 2024-04-24T22:44:53.346 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38619.log 2024-04-24T22:44:53.346 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38612.log.gz 2024-04-24T22:44:53.347 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37864.log 2024-04-24T22:44:53.347 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38940.log.gz 2024-04-24T22:44:53.348 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42374.log 2024-04-24T22:44:53.348 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38619.log.gz 2024-04-24T22:44:53.349 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44425.log 2024-04-24T22:44:53.349 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37864.log.gz 2024-04-24T22:44:53.350 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37820.log 2024-04-24T22:44:53.350 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42374.log.gz 2024-04-24T22:44:53.351 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44425.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-24T22:44:53.351 INFO:teuthology.orchestra.run.smithi047.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44425.log.gz 2024-04-24T22:44:53.352 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43574.log 2024-04-24T22:44:53.352 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37820.log.gz 2024-04-24T22:44:53.353 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38933.log 2024-04-24T22:44:53.354 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42332.log 2024-04-24T22:44:53.355 INFO:teuthology.orchestra.run.smithi047.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43574.log.gz 2024-04-24T22:44:53.356 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42118.log 2024-04-24T22:44:53.356 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38933.log.gz 2024-04-24T22:44:53.357 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42893.log 2024-04-24T22:44:53.357 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42332.log.gz 2024-04-24T22:44:53.358 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42118.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42118.log.gz 2024-04-24T22:44:53.359 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41240.log 2024-04-24T22:44:53.360 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42893.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42893.log.gz 2024-04-24T22:44:53.360 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39677.log 2024-04-24T22:44:53.361 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38395.log 2024-04-24T22:44:53.361 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41240.log.gz 2024-04-24T22:44:53.361 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39677.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39677.log.gz 2024-04-24T22:44:53.362 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43490.log 2024-04-24T22:44:53.363 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40904.log 2024-04-24T22:44:53.363 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38395.log.gz 2024-04-24T22:44:53.364 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43490.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43490.log.gz 2024-04-24T22:44:53.364 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44667.log 2024-04-24T22:44:53.365 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45706.log 2024-04-24T22:44:53.365 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40904.log.gz 2024-04-24T22:44:53.366 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44667.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44667.log.gz 2024-04-24T22:44:53.367 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38955.log 2024-04-24T22:44:53.368 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40709.log 2024-04-24T22:44:53.368 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45706.log.gz 2024-04-24T22:44:53.369 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38955.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38955.log.gz 2024-04-24T22:44:53.369 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-24T22:44:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.40709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40709.log.gz 2024-04-24T22:44:53.370 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41851.log 2024-04-24T22:44:53.371 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46040.log 2024-04-24T22:44:53.373 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37307.log 2024-04-24T22:44:53.373 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41851.log.gz 2024-04-24T22:44:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45956.log 2024-04-24T22:44:53.374 INFO:teuthology.orchestra.run.smithi047.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46040.log.gz 2024-04-24T22:44:53.375 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37307.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-24T22:44:53.376 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37307.log.gz 2024-04-24T22:44:53.377 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45956.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.45956.log.gz 2024-04-24T22:44:53.377 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-24T22:44:53.378 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42164.log 2024-04-24T22:44:53.380 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42058.log 2024-04-24T22:44:53.381 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37914.log 2024-04-24T22:44:53.382 INFO:teuthology.orchestra.run.smithi047.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42164.log.gz 2024-04-24T22:44:53.383 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43167.log 2024-04-24T22:44:53.383 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42058.log.gz 2024-04-24T22:44:53.384 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37914.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37914.log.gz 2024-04-24T22:44:53.385 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42016.log 2024-04-24T22:44:53.385 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43167.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43167.log.gz 2024-04-24T22:44:53.386 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37998.log 2024-04-24T22:44:53.387 INFO:teuthology.orchestra.run.smithi047.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-24T22:44:53.387 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42016.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-24T22:44:53.388 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42016.log.gz 2024-04-24T22:44:53.389 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38311.log 2024-04-24T22:44:53.389 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37998.log.gz 2024-04-24T22:44:53.390 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39975.log 2024-04-24T22:44:53.392 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33686.log 2024-04-24T22:44:53.392 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38311.log.gz 2024-04-24T22:44:53.393 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39975.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25563.log 2024-04-24T22:44:53.393 INFO:teuthology.orchestra.run.smithi047.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.39975.log.gz 2024-04-24T22:44:53.394 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41030.log 2024-04-24T22:44:53.394 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33686.log.gz 2024-04-24T22:44:53.395 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.tmp-client.admin.25563.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41590.log 2024-04-24T22:44:53.396 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.25563.log.gz 2024-04-24T22:44:53.397 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41505.log 2024-04-24T22:44:53.397 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41030.log.gz 2024-04-24T22:44:53.398 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41590.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.37677.log 2024-04-24T22:44:53.398 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41590.log.gz 2024-04-24T22:44:53.400 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45244.log 2024-04-24T22:44:53.400 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41505.log.gz 2024-04-24T22:44:53.401 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44551.log 2024-04-24T22:44:53.401 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37677.log.gz 2024-04-24T22:44:53.402 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45244.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45244.log.gz 2024-04-24T22:44:53.403 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42416.log 2024-04-24T22:44:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44551.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44551.log.gz 2024-04-24T22:44:53.404 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45496.log 2024-04-24T22:44:53.406 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33302.log 2024-04-24T22:44:53.406 INFO:teuthology.orchestra.run.smithi047.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.42416.log.gz 2024-04-24T22:44:53.407 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42206.log 2024-04-24T22:44:53.407 INFO:teuthology.orchestra.run.smithi047.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45496.log.gz 2024-04-24T22:44:53.409 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43338.log 2024-04-24T22:44:53.409 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33302.log.gz 2024-04-24T22:44:53.410 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42809.log 2024-04-24T22:44:53.410 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42206.log.gz 2024-04-24T22:44:53.412 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43744.log 2024-04-24T22:44:53.412 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43338.log.gz 2024-04-24T22:44:53.413 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42809.log: gzip -5 --verbose -- 90.2% -- replaced with /var/log/ceph/ceph-client.admin.42809.log.gz 2024-04-24T22:44:53.413 INFO:teuthology.orchestra.run.smithi047.stderr: /var/log/ceph/ceph-client.admin.37956.log 2024-04-24T22:44:53.414 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43744.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.43744.log.gz 2024-04-24T22:44:53.415 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41633.log 2024-04-24T22:44:53.416 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40988.log 2024-04-24T22:44:53.416 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37956.log.gz 2024-04-24T22:44:53.417 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41633.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41633.log.gz 2024-04-24T22:44:53.419 INFO:teuthology.orchestra.run.smithi047.stderr:gzip/var/log/ceph/ceph-client.admin.40988.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38142.log 2024-04-24T22:44:53.419 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz 2024-04-24T22:44:53.420 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45622.log 2024-04-24T22:44:53.422 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38142.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-24T22:44:53.422 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38142.log.gz 2024-04-24T22:44:53.423 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45622.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45622.log.gz 2024-04-24T22:44:53.423 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38227.log 2024-04-24T22:44:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39193.log 2024-04-24T22:44:53.426 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38227.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38227.log.gz 2024-04-24T22:44:53.427 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39193.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39193.log.gz 2024-04-24T22:44:53.428 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38943.log 2024-04-24T22:44:53.430 INFO:teuthology.orchestra.run.smithi113.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-24T22:44:53.431 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38943.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38943.log.gz 2024-04-24T22:44:53.431 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44932.log 2024-04-24T22:44:53.432 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38950.log 2024-04-24T22:44:53.433 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44932.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44932.log.gz 2024-04-24T22:44:53.435 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38269.log 2024-04-24T22:44:53.437 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41198.log 2024-04-24T22:44:53.437 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38950.log.gz 2024-04-24T22:44:53.438 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42681.log 2024-04-24T22:44:53.439 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38269.log: /var/log/ceph/ceph-client.admin.41198.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38269.log.gz 2024-04-24T22:44:53.441 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41198.log.gz/var/log/ceph/ceph-client.admin.42681.log: 2024-04-24T22:44:53.442 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42681.log.gz 2024-04-24T22:44:53.442 INFO:teuthology.orchestra.run.smithi047.stderr: /var/log/ceph/ceph-client.admin.39235.log 2024-04-24T22:44:53.442 INFO:teuthology.orchestra.run.smithi113.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-24T22:44:53.444 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44709.log 2024-04-24T22:44:53.445 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39235.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39235.log.gz 2024-04-24T22:44:53.446 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44709.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44709.log.gz 2024-04-24T22:44:53.447 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43532.log 2024-04-24T22:44:53.448 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37726.log 2024-04-24T22:44:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43532.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43532.log.gz 2024-04-24T22:44:53.451 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.37726.log: /var/log/ceph/ceph-client.admin.39635.log 2024-04-24T22:44:53.451 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37726.log.gz 2024-04-24T22:44:53.452 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45412.log 2024-04-24T22:44:53.454 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39635.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39635.log.gz 2024-04-24T22:44:53.454 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43252.log 2024-04-24T22:44:53.455 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45412.log.gz 2024-04-24T22:44:53.457 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38437.log 2024-04-24T22:44:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43252.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43252.log.gz 2024-04-24T22:44:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:gzip/var/log/ceph/ceph-client.admin.38437.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44341.log 2024-04-24T22:44:53.460 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38437.log.gz 2024-04-24T22:44:53.461 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44383.log 2024-04-24T22:44:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44341.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44341.log.gz 2024-04-24T22:44:53.463 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45664.log/var/log/ceph/ceph-client.admin.44383.log: 2024-04-24T22:44:53.464 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44383.log.gz 2024-04-24T22:44:53.465 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45790.log 2024-04-24T22:44:53.466 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45664.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45664.log.gz 2024-04-24T22:44:53.467 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43079.log 2024-04-24T22:44:53.468 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45790.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45790.log.gz 2024-04-24T22:44:53.469 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41156.log 2024-04-24T22:44:53.470 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43079.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43079.log.gz 2024-04-24T22:44:53.471 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44625.log 2024-04-24T22:44:53.472 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41156.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41156.log.gz 2024-04-24T22:44:53.472 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45914.log 2024-04-24T22:44:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44625.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44625.log.gz 2024-04-24T22:44:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41463.log 2024-04-24T22:44:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45914.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45914.log.gz 2024-04-24T22:44:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41463.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.41463.log.gz 2024-04-24T22:44:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44793.log 2024-04-24T22:44:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38041.log 2024-04-24T22:44:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44793.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44793.log.gz 2024-04-24T22:44:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-04-24T22:44:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38041.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38041.log.gz 2024-04-24T22:44:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40194.log 2024-04-24T22:44:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mds.e.log: /var/log/ceph/ceph-client.admin.40194.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.40194.log.gz 2024-04-24T22:44:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41072.log 2024-04-24T22:44:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45058.log 2024-04-24T22:44:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42458.log 2024-04-24T22:44:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45058.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45058.log.gz 2024-04-24T22:44:53.495 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41072.log.gz 2024-04-24T22:44:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38604.log 2024-04-24T22:44:53.497 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38604.log: /var/log/ceph/ceph-client.admin.42458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33567.log 2024-04-24T22:44:53.499 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42458.log.gz 2024-04-24T22:44:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39807.log 2024-04-24T22:44:53.501 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33567.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33567.log.gz 2024-04-24T22:44:53.502 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38604.log.gz 2024-04-24T22:44:53.504 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39807.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39807.log.gz 2024-04-24T22:44:53.504 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39593.log 2024-04-24T22:44:53.507 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43023.log 2024-04-24T22:44:53.508 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39593.log.gz 2024-04-24T22:44:53.509 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39373.log 2024-04-24T22:44:53.514 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43023.log: /var/log/ceph/ceph-client.admin.39373.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37776.log 2024-04-24T22:44:53.514 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43023.log.gz 2024-04-24T22:44:53.515 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39373.log.gz 2024-04-24T22:44:53.517 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45142.log 2024-04-24T22:44:53.522 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45142.log: /var/log/ceph/ceph-client.admin.37776.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41687.log 2024-04-24T22:44:53.523 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37776.log.gz 2024-04-24T22:44:53.523 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45142.log.gz 2024-04-24T22:44:53.526 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43786.log 2024-04-24T22:44:53.526 INFO:teuthology.orchestra.run.smithi047.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-04-24T22:44:53.527 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39151.log 2024-04-24T22:44:53.530 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41687.log: 91.7%/var/log/ceph/ceph-client.admin.43786.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.43786.log.gz 2024-04-24T22:44:53.530 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38353.log 2024-04-24T22:44:53.530 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39151.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39151.log.gz 2024-04-24T22:44:53.531 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41687.log.gz 2024-04-24T22:44:53.532 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38618.log 2024-04-24T22:44:53.533 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38353.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38353.log.gz 2024-04-24T22:44:53.534 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38601.log 2024-04-24T22:44:53.535 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38618.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38618.log.gz 2024-04-24T22:44:53.536 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38617.log 2024-04-24T22:44:53.537 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38601.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38601.log.gz 2024-04-24T22:44:53.537 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39509.log 2024-04-24T22:44:53.539 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38617.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38617.log.gz 2024-04-24T22:44:53.539 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37521.log 2024-04-24T22:44:53.541 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39509.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43659.log/var/log/ceph/ceph-client.admin.37521.log: 2024-04-24T22:44:53.542 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39509.log.gz 2024-04-24T22:44:53.542 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37521.log.gz 2024-04-24T22:44:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45286.log 2024-04-24T22:44:53.545 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43659.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43659.log.gz 2024-04-24T22:44:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44160.log 2024-04-24T22:44:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45286.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45286.log.gz 2024-04-24T22:44:53.547 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40362.log 2024-04-24T22:44:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44160.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44160.log.gz 2024-04-24T22:44:53.550 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44299.log/var/log/ceph/ceph-client.admin.40362.log: 2024-04-24T22:44:53.550 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40362.log.gz 2024-04-24T22:44:53.551 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39933.log 2024-04-24T22:44:53.553 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44299.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44299.log.gz 2024-04-24T22:44:53.554 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44467.log 2024-04-24T22:44:53.554 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39933.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39933.log.gz 2024-04-24T22:44:53.555 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38954.log 2024-04-24T22:44:53.557 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44467.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44467.log.gz 2024-04-24T22:44:53.558 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5/var/log/ceph/ceph-client.admin.38954.log: --verbose -- /var/log/ceph/ceph-client.admin.44056.log 2024-04-24T22:44:53.558 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38954.log.gz 2024-04-24T22:44:53.559 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45832.log 2024-04-24T22:44:53.561 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44056.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44056.log.gz 2024-04-24T22:44:53.562 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45832.log: /var/log/ceph/ceph-client.admin.43972.log 2024-04-24T22:44:53.563 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45832.log.gz 2024-04-24T22:44:53.564 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-24T22:44:53.565 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43972.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz 2024-04-24T22:44:53.566 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41771.log/var/log/ceph/ceph-mds.a.log: 2024-04-24T22:44:53.568 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38185.log 2024-04-24T22:44:53.570 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41771.log.gz 2024-04-24T22:44:53.572 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39467.log 2024-04-24T22:44:53.574 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38185.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38185.log.gz 2024-04-24T22:44:53.577 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42290.log 2024-04-24T22:44:53.579 INFO:teuthology.orchestra.run.smithi047.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42723.log 2024-04-24T22:44:53.580 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39467.log.gz 2024-04-24T22:44:53.582 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42290.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42290.log.gz 2024-04-24T22:44:53.583 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45872.log 2024-04-24T22:44:53.588 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42723.log: /var/log/ceph/ceph-client.admin.45872.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.42723.log.gz 2024-04-24T22:44:53.589 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45016.log 2024-04-24T22:44:53.592 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45016.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45872.log.gz 2024-04-24T22:44:53.592 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44203.log 2024-04-24T22:44:53.595 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39765.log 2024-04-24T22:44:53.596 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44203.log: -- replaced with /var/log/ceph/ceph-client.admin.45016.log.gz 2024-04-24T22:44:53.599 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39765.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44203.log.gz 2024-04-24T22:44:53.599 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39765.log.gz 2024-04-24T22:44:53.599 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41547.log 2024-04-24T22:44:53.601 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41379.log 2024-04-24T22:44:53.603 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42935.log 2024-04-24T22:44:53.603 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2%/var/log/ceph/ceph-client.admin.41379.log: -- replaced with /var/log/ceph/ceph-client.admin.41547.log.gz 2024-04-24T22:44:53.605 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41379.log.gz 2024-04-24T22:44:53.605 INFO:teuthology.orchestra.run.smithi047.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-24T22:44:53.607 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42935.log: -- /var/log/ceph/ceph-client.admin.41729.log 2024-04-24T22:44:53.608 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42935.log.gz 2024-04-24T22:44:53.609 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45100.log 2024-04-24T22:44:53.610 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41729.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41729.log.gz 2024-04-24T22:44:53.611 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40946.log 2024-04-24T22:44:53.612 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45100.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45100.log.gz 2024-04-24T22:44:53.613 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40946.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40946.log.gz 2024-04-24T22:44:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43381.log 2024-04-24T22:44:53.615 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40862.log 2024-04-24T22:44:53.617 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43381.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.43381.log.gz 2024-04-24T22:44:53.618 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43125.log 2024-04-24T22:44:53.618 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40862.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40862.log.gz 2024-04-24T22:44:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43210.log 2024-04-24T22:44:53.621 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43125.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43125.log.gz 2024-04-24T22:44:53.621 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44509.log 2024-04-24T22:44:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45580.log 2024-04-24T22:44:53.623 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43210.log.gz 2024-04-24T22:44:53.625 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44509.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44509.log.gz 2024-04-24T22:44:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45580.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45580.log.gz 2024-04-24T22:44:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38952.log 2024-04-24T22:44:53.628 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38616.log 2024-04-24T22:44:53.629 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38952.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38952.log.gz 2024-04-24T22:44:53.630 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37568.log 2024-04-24T22:44:53.631 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38616.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38616.log.gz 2024-04-24T22:44:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37568.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37568.log.gz 2024-04-24T22:44:53.640 INFO:teuthology.orchestra.run.smithi047.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-24T22:44:53.773 INFO:teuthology.orchestra.run.smithi047.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-24T22:44:53.787 INFO:teuthology.orchestra.run.smithi047.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-24T22:44:53.787 INFO:teuthology.orchestra.run.smithi113.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-24T22:44:53.844 INFO:teuthology.orchestra.run.smithi113.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-24T22:44:53.931 INFO:teuthology.orchestra.run.smithi047.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-24T22:44:53.946 INFO:teuthology.orchestra.run.smithi113.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-24T22:44:53.994 INFO:teuthology.orchestra.run.smithi113.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-24T22:44:53.996 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-24T22:44:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m0.736s 2024-04-24T22:44:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m2.828s 2024-04-24T22:44:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m0.126s 2024-04-24T22:44:54.110 INFO:teuthology.orchestra.run.smithi047.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-24T22:44:54.156 INFO:teuthology.orchestra.run.smithi047.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-24T22:44:54.159 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-24T22:44:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m0.899s 2024-04-24T22:44:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:user 0m3.066s 2024-04-24T22:44:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m0.826s 2024-04-24T22:44:54.160 INFO:tasks.ceph:Archiving logs... 2024-04-24T22:44:54.161 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135/remote/smithi047/log 2024-04-24T22:44:54.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T22:44:54.773 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135/remote/smithi113/log 2024-04-24T22:44:54.774 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T22:44:55.183 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T22:44:55.192 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T22:44:55.193 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T22:44:55.201 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T22:44:55.339 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T22:44:55.348 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T22:44:55.349 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:44:55.356 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T22:44:55.371 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:44:55.374 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:44:55.375 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-04-24T22:44:55.375 INFO:teuthology.orchestra.run.smithi113.stdout:^- hv01.front.sepia.ceph.com 2 6 377 58 +1215us[+1215us] +/- 591ms 2024-04-24T22:44:55.375 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -122us[ -111us] +/- 60ms 2024-04-24T22:44:55.375 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +164us[ +164us] +/- 72ms 2024-04-24T22:44:55.375 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:44:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-04-24T22:44:55.418 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T22:44:55.418 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-24T22:44:55.418 INFO:teuthology.orchestra.run.smithi047.stdout:^- hv01.front.sepia.ceph.com 2 6 377 57 +1195us[+1192us] +/- 591ms 2024-04-24T22:44:55.418 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv02.front.sepia.ceph.com 3 6 377 57 -135us[ -139us] +/- 60ms 2024-04-24T22:44:55.418 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +144us[ +140us] +/- 72ms 2024-04-24T22:44:55.418 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T22:44:55.419 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T22:44:55.430 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T22:44:55.432 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T22:44:55.442 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T22:44:55.649 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T22:44:55.709 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:44:55.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:44:55.810 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:44:55.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:44:55.837 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T22:44:55.847 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T22:44:55.868 INFO:teuthology.task.internal:Duration was 836.328279 seconds 2024-04-24T22:44:55.868 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T22:44:55.885 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T22:44:55.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T22:44:55.893 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T22:44:55.931 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:44:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:44:56.256 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T22:44:56.256 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-04-24T22:44:56.256 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T22:44:56.334 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-04-24T22:44:56.335 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T22:44:56.362 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T22:44:56.362 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T22:44:56.368 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T22:44:56.450 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T22:44:56.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T22:44:56.455 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T22:44:56.630 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T22:44:56.642 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T22:44:56.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T22:44:56.728 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T22:44:56.752 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T22:44:56.763 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T22:44:56.770 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T22:44:56.813 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-04-24T22:44:56.869 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-04-24T22:44:56.916 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T22:44:56.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:44:56.966 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T22:44:56.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:44:56.980 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T22:44:56.991 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T22:44:56.992 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135/remote/smithi047 2024-04-24T22:44:56.992 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T22:44:57.092 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672135/remote/smithi113 2024-04-24T22:44:57.092 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T22:44:57.124 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T22:44:57.124 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T22:44:57.130 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T22:44:57.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T22:44:57.195 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T22:44:57.195 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T22:44:57.246 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T22:44:57.246 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T22:44:57.254 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T22:44:57.268 INFO:teuthology.orchestra.run.smithi113.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 22:44 /home/ubuntu/cephtest 2024-04-24T22:44:57.307 INFO:teuthology.orchestra.run.smithi047.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 22:44 /home/ubuntu/cephtest 2024-04-24T22:44:57.310 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T22:44:57.322 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T22:44:57.374 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} duration: 836.3282785415649 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-24T22:44:57.374 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T22:44:57.464 INFO:teuthology.run:pass